Over time, my photo collection has grown to about 15,000 pictures.  Because my principal disk has usually not been big enough for the entire collection, it has come to be duplicated in part across several external disks.   Over the years, I’ve also accumulated downsampled, smaller versions of some of the pictures, as well as edited versions.

Now that my laptop drive is big enough to hold all my pictures comfortable, I’ve been meaning for a to coalesce these several collections, getting rid of duplicates, and keeping the some originals, and the highest-res and edited versions.   Until now, I’ve shied away from performing this merge and de-dup, but finally got it done this weekend.  As I haven’t been able to find much written about how to do this, I’m writing it up here.  It isn’t for the faint of heart, but it is doable and worked for me.  All my pictures are now de-duped and on my MacBook drive.

Practice on a subset of your duplicated library first to master the techniques involved.

Problems:

1.multiple overlapping photo collections

2.on multiple disks

3.some “imported” some on the drive as OsX aliases

4.scanned pictures (same creation date)

5.exact copies present

6.down-res copies present

7.desired “corrections”

8.thumbnails of lib contents themselves in library

Goal:

•one merged iPhoto library with all images on 1 disk with no duplicates

•keeping hand selected versions of the “near” duplicates

•no thumbnails

Tools:

•AliasHerder (free, [http://www.rorohiko.com/wordpress/downloads/aliasherder/)

•Duplicate Anhilator (7.95, http://www.brattoo.com/propaganda/)

•

         These are procedures which you’ll need to use time and again when manipulating the iPhoto libs to get rid of dups.

        Fundamental Techniques used:

1. 1.iPhoto: create a disk-based photo collection
   

        i. select all photos in a library and Export as original resolution   

2. 2.iPhoto: open an alternative library.  Hold option down while starting iPhoto, it will give you the option of opening an alternative library.
   

3. 3.
   

4. 4.iPhoto: delete from Library all photos in a selected album
   i. CMD-K (keyword window). Create a “delete” keyword
   

        ii. select the delete keyword to tag all pics

        iii. select Library, search tag “delete”

        iv. Move To Trash

3.iPhoto: create an album from tags: use search to find, them CMD-A drag to a result album.  Use SMART ALBUMS to create an ALL or ANY combinations of tags.

4.iPhoto: delete from an album all photos you wish to delete, then delete those from Lib :

i.open album

ii.CMD-K (keyword window).  create a “delete” keyword

iii.CMD-A select all, click “delete” keyword to add it to all photos

iv.delete from album all photos you want to get rid of

v.CMD-A select all remaining.  click on “delete” keyword to untag the remaining ones

vi.Click on Library, Search for “delete” keyword over all photos.

vii.Move these to trash

5.DA: find “near” duplicates: Use Creation date and 8 chars of filename

Steps:

1.Prep: copy/backup destination iPhoto set

NB: You can use iPhoto Library Manager (http://www.fatcatsoftware.com/iplm/, $20) to merge your various libraries and skip 2 and 3.

2.Prep: create collections from iPhoto sets (needed only if there are some collections which are in iPhoto with pictures which have been modified, and you don’t want the originals)

i.select the iPhoto library desired using option-start technique

ii.select Library/Photos in left pane

iii.select all pictures CMD-A

iv.File>Export Kind=Original

v.create a new picture directory

3.Merge collections into one iPhoto Library

i.open iPhoto destination library (option-start?)

ii.make sure pictures will be copied, by setting Preferences>Advanced>Copy items to the iPhoto Library

iii.File>Import to Library… each collection

4.Collect all photos onto destination disk

i.exit iPhoto

ii.open AliasHerder, right click “Keep in Dock” on AliasHerder dock icon.  close Alias Herder (stays in dock)

iii.find iPhoto file directories in Finder by right-click Show Package Contents on the iPhoto library (in Pictures>iPhoto Library in Finder)

iv.For each of Data, Originals, and Modified

v.drag the file to AliasHerder dock icon, with “option” depressed

1.wait while aliases are chased down and files copied

2.if this is very quick (under 10secs), probably didn’t hold down the option key while dragging, try again.

5.Remove thumbnails

i.Run Thumbnail Annihilator (comes with Dup Annihilator) with Preferences Move Thumbnails to trash

6.Remove exact duplicates

i.Run DA with MD5 and select move to trash

7.Remove “near” duplicates

1. i.Isolate near duplicates in dupsort: run Duplicate Annhilator over entire Library with preferences set to 8 chars and creation or Exif creation date, and tag dups with “duplicate”
   

ii.Search/Filter duplicate in dupsort, sort by title

3. iii.copy contents of smart album to a new album, called “dupsort”, sort by title, CMD-A and flag, clear find
   

4. iv.should now have all potential dups, and found dups will be flagged.
   

5. v.CMD-A and tag all with deleted.
   

5. v.for each obvious orig/mod pair delete one to be discarded from album
   

6. vi.for each obvious want to delete, delete from album
   

7. vii. select all, click on “delete” tag to remove, go to Library, select All, Filter by “delete” tag, move to Trash
   

ix.delete “downsampled” images

1.in my case search and move to downsampled all _01, _02

2.delete all downsampled using delet tag trick

Notes:

• “imported” iPhoto sets will lose iPhoto attributes, faces, places, events, original/modified status etc, UNLESS you use iPhoto Library Manager to merge the libraries at the onset.

•a number of these techniques use the Description fields, which means you’ll overwrite your descriptions if you’re using them.

Friday, May 8, 2009