Add `sync` command to sync metadata between shared libraries

[RhetTbull]

  With the release of macOS Ventura, there is a new feature called [iCloud Shared Photo Library](iCloud Shared Photo Library). It is convenient and does a good job of synchronizing videos and photos who are granted access as "Participants".  The one issue I have with this new feature is it doesn't synchronize the folders and albums. Could OSXPhotos help with this?

Here is the situation that I'm trying to solve:

1. My wife and I share an iCloud Shared Photo Library
2. We have moved 100% of our photos on our film roll to be shared, and therefore we have 100% matching photos in our "Shared Library."
3. My wife does 100% of the organization of photos on her iPhone and MacBook.
4. We would like to keep the same organization system using folders and albums in Photos.

A possible approach to solve this using OSXPhotos:

1. Using OSXPhotos, download in JSON format the folders and albums from my wife's Photos database.
2. The JSON file is saved to a shared iCloud file folder.
3. Using OSXPhotos, read from the shared iCloud file folder the JSON file and apply it to my Photos database.
4. Given that the photos are identical in the shared libraries, the folder & album organization should organize my Photos database.

Would it be possible to perform this synchronization with OSXPhotos? If so, what is the command line syntax? If not, could this be added as a feature to OSXPhotos?

Originally posted by @qkeddy in https://github.com/RhetTbull/osxphotos/discussions/886

[RhetTbull]

It is not currently possible to do this in osxphotos but the workflow you described should be relatively easy to implement. I really like this idea as a feature though and will open a new issue to add this.

I envision something like a sync-shared command:

On computer 1: osxphotos sync-shared --export /path/to/shared/folder/computer1.db

Then on computer 2: osxphotos sync-shared --import /path/to/shared/folder/computer1.db --album --title --description --keywords

You could specify which metadata to sync. (Does the shared library currently sync other metadata like titles and keywords?)

I've already built a similar script for a user who did not use iCloud but wanted to sync metadata / albums between iPhone and Mac. In that case, the difficult part was matching the photos as there wasn't a unique identifier to correlate the two libraries. In a regular iCloud library, each photo has a unique cloud identifier (ZCLOUDMASTER.ZCLOUDMASTERGUID). For photos in a shared library, I'm not sure how Apple uniquely identifies the photos shared across two different libraries so that will have to be figured out. I have a machine running Ventura but have not used the shared library feature yet so don't have data to test on.

edit: Rather than JSON, I'd probably use a sqlite database as the file format as it would be more performant and flexible (and still accessible via a variety of tools)

[RhetTbull]

@all-contributors please add @qkeddy for ideas, data

[allcontributors[bot]]

@RhetTbull

I've put up a pull request to add @qkeddy! :tada:

[qkeddy]

Thanks @RhetTbull . Looking forward to helping out on this project.

[rizwank]

Oh, this would be cool. Sounds it'd be like a one way sync? Only for certain albums?

[RhetTbull]

@rizwank I've been working on the sync command on a separate branch and it's about 70-80% done. It will work both ways and will have the option to sync some or all photos between two separate libraries. It will work even for non-shared libraries -- for example, some people manually sync their phones to different libraries (not via iCloud) or copy photos between libraries. It will also allow you to sync data from the iPhone to the Mac (but not the other way) for the case where the user doesn't use iCloud but wants to copy favorites, etc. from the iPhone. You'll be able to select which metadata you want to sync, for example, "only favorites", or "favorites, keywords, and albums".

Here's an example command line (subject to change as I'm still working on the code) that allows you to perform a two way sync (when run from each Mac) assuming you're storing the exported data on a shared network drive (for example, a shared iCloud folder):

osxphotos sync --export /path/to/iCloud/folder/computer1.db --import /path/to/iCloud/folder/computer2.db --merge albums,favorites

[RhetTbull]

Here's the current help text for the in-development sync command. (Excluding all the usual query/filter options common to osxphotos query and osxphotos export):

Usage: osxphotos sync [OPTIONS]

  Sync metadata and albums between Photos libraries

Options:
  -e, --export EXPORT_FILE        Export metadata to file EXPORT_FILE for
                                  later use with --import.
  -i, --import IMPORT_PATH        Import metadata from file IMPORT_PATH.
                                  IMPORT_PATH can a Photos library, a Photos
                                  database, or a metadata export file created
                                  with --export.
  -s, --set METADATA              When used with --import, set metadata in
                                  local Photos library to match import data.
                                  Multiple metadata properties can be
                                  specified by repeating the --set option or
                                  by using a comma-separated list. METADATA
                                  can be one of: all, keywords, albums, title,
                                  description, favorite. For example, to set
                                  keywords and favorite, use `--set keywords
                                  --set favorite` or `--set
                                  keywords,favorite`. If `--set all` is
                                  specified, all metadata will be set. Note
                                  that using --set overwrites any existing
                                  metadata in the local Photos library. For
                                  example, if a photo is marked as favorite in
                                  the local library but not in the import
                                  source, --set favorite will clear the
                                  favorite status in the local library. The
                                  exception to this is that `--set album` will
                                  not remove the photo from any existing
                                  albums in the local library but will add the
                                  photo to any new albums specified in the
                                  import source.See also --merge.
  -m, --merge METADATA            When used with --import, merge metadata in
                                  local Photos library with import data.
                                  Multiple metadata properties can be
                                  specified by repeating the --merge option or
                                  by using a comma-separated list. METADATA
                                  can be one of: all, keywords, albums, title,
                                  description, favorite. For example, to merge
                                  keywords and favorite, use `--merge keywords
                                  --merge favorite` or `--merge
                                  keywords,favorite`. If `--merge all` is
                                  specified, all metadata will be merged. Note
                                  that using --merge does not overwrite any
                                  existing metadata in the local Photos
                                  library. For example, if a photo is marked
                                  as favorite in the local library but not in
                                  the import source, --merge favorite will not
                                  change the favorite status in the local
                                  library. See also --set.
  --report REPORT_FILE            Write a report of all photos that were
                                  processed with --import. The extension of
                                  the report filename will be used to
                                  determine the format. Valid extensions are:
                                  .csv (CSV file), .json (JSON), .db and
                                  .sqlite (SQLite database). REPORT_FILE may
                                  be a an osxphotos template string, for
                                  example, --report 'update_{today.date}.csv'
                                  will write a CSV report file named with
                                  today's date. See also --append.
  --append                        If used with --report, add data to existing
                                  report file instead of overwriting it. See
                                  also --report.
  --dry-run                       Dry run; when used with --import, don't
                                  actually update metadata.
  -V, --verbose                   Print verbose output.
  -T, --timestamp                 Add time stamp to verbose output.

[RhetTbull]

Initial version of osxphotos sync command released in v0.56.1. This will probably work with iCloud Shared Libraries but I've not been able to test it. osxphotos cannot currently identify photos that are included in an iCloud Shared Library but it can identify identical photos across different libraries so I believe the sync command is still usable with iCloud Shared Library. Once I get more test data I'll be able to add a filter for those photos included in the Shared Library.

If you want to give this try, read the docs: osxphotos help sync and post any questions here. I recommend running with --dry-run first. There's also a --report option that's useful for identifying what changed.

[RhetTbull]

One caveat that I need to update in the documentation: currently only top-level albums are synced. This is simply a limitation of the "min viable product" for the initial release and something I'll fix eventually.