Shared photos not working on Monterey

[neebah]

Error exporting photo (50098B3B-5442-47AA-8F8C-B0BF0270965E: IMG_6467.HEIC) as "location": Invalid photo id: 50098B3B-5442-47AA-8F8C-B0BF0270965E (_photoinfo_export.py: 1343)

Get this error even though all pictures are loaded into the library.

[RhetTbull]

What's the full command line you're using with osxphotos?

[neebah]

osxphotos export --download-missing --directory {folder_album} --not-shared --deleted "export_location"

I started doing the not-shared to see if it had to do with any of the share libraries, but that wasn't it.

[RhetTbull]

Are you getting lots of these errors or only a few?

I think this is because --download-missing doesn't work with deleted files and you're also using --deleted. What should happen in this case is all other photos will be exported but those in the trash and are missing will not be exported and will produce this error. Photos does not sync the actual image from iCloud for photos in the trash so osxphotos cannot export these. Photos recently deleted on the local machine will still be available in the trash and those can be exported.

Try this: osxphotos query --json --uuid 50098B3B-5442-47AA-8F8C-B0BF0270965E

And look for the value that says "intrash": true or "intrash": false. My guess is it'll say "intrash": true for that particular photo.

[neebah]

I'll give that a look after the export. Yes I'm getting many of those. Should I run an export with the --download-missing and then an update export with --deleted

[neebah]

Just looking at this point. There is only one photo in the Recently Deleted spot in photos

[RhetTbull]

I'll give that a look after the export. Yes I'm getting many of those. Should I run an export with the --download-missing and then an update export with --deleted

That might work if indeed the issue is deleted photos. But if you only have one deleted photo it might be something else. Try the export with the --use-photokit flag with your current options which uses a different code path for missing photos. It's usually more robust than just using --download-missing by itself which defaults to AppleScript.

[neebah]

Just so I'm clear try my export as given to you with --use-photokit instead of --download-missing. Should I leave the --deleted out

[RhetTbull]

Just so I'm clear try my export as given to you with --use-photokit instead of --download-missing. Should I leave the --deleted out

I would try this: osxphotos export --download-missing --use-photokit --directory {folder_album} --not-shared --deleted "export_location" --update

Using --use-photokit with --download-missing causes osxphotos to use PhotoKit (a native macOS code framework) instead of AppleScript to download missing photos from iCloud. The addition of --update will cause osxphotos to only export photos not previously exported to save time. You'll likely still get errors for the deleted photo(s) but shouldn't get a bunch of these if you don't have a bunch of deleted images.

[neebah]

So no error messages, but the same amount of photos were reported missing after the export

[neebah]

Ran an update export without the --not-shared and got this error message: Same number of missing

Error exporting photo (093B5199-93D5-4925-9F2C-60FABED78500: 6F39365D-16BB-48F1-B440-24B849CCDFC9.JPG) as 6F39365D-16BB-48F1-B440-24B849CCDFC9.JPG: expected str, bytes or os.PathLike object, not NoneType

[neebah]

ran an update without the --not-shared and without the --download-missing

Got less errors this time, but of this type

Error exporting photo (08DE40C8-A47A-427F-95D6-15430C6FEE80: IMG_6470.HEIC) as IMG_6470.HEIC: expected str, bytes or os.PathLike object, not NoneType

[neebah]

also seeing this error. I'm confident this relates to photos in the sharing library

Error exporting photo (ECBEEA11-AF25-41B4-B63C-763F76B330C8: 6E81D59F-8A6D-4A1F-BF97-319F5FAB9507.mp4) as "export dest" couldn’t be copied to “export-folder” because an item with the same name already exists." UserInfo={NSSourceFilePathErrorKey="file location", NSUserStringVariant=( Copy ), NSDestinationFilePath="export location", NSFilePath="export dest", NSUnderlyingError=0x600003e843f0 {Error Domain=NSPOSIXErrorDomain Code=17 "File exists"}} (_photoinfo_export.py: 1274)

[RhetTbull]

Error exporting photo (ECBEEA11-AF25-41B4-B63C-763F76B330C8: 6E81D59F-8A6D-4A1F-BF97-319F5FAB9507.mp4) as "export dest" couldn’t be copied to “export-folder” because an item with the same name already exists." UserInfo={NSSourceFilePathErrorKey="file location", NSUserStringVariant=( Copy ), NSDestinationFilePath="export location", NSFilePath="export dest", NSUnderlyingError=0x600003e843f0 {Error Domain=NSPOSIXErrorDomain Code=17 "File exists"}} (_photoinfo_export.py: 1274)

This error occurs if there's a name collision in the export folder (osxphotos tries to export two photos with the same name to the same folder) though in most cases, osxphotos should try to increment the file name, e.g. add (1), (2) etc to the name. Not sure why it's not happening in your case so possibly a bug.

It would be helpful to have some more debugging info on the particular photos that are causing these errors so I can see if there's something in common with them. The long alpha-numeric string that shows up in each of these error messages (e.g. ECBEEA11-AF25-41B4-B63C-763F76B330C8) is the unique ID (UUID) by which Photos tracks the photo. Using that, osxphotos can extract additional info about the photo.

Could you try the following command: osxphotos debug-dump --dump photos --uuid 50098B3B-5442-47AA-8F8C-B0BF0270965E --uuid ECBEEA11-AF25-41B4-B63C-763F76B330C8 --uuid 08DE40C8-A47A-427F-95D6-15430C6FEE80 --uuid 093B5199-93D5-4925-9F2C-60FABED78500 > debug.txt

and post the output here. I recommend you inspect the debug.txt file first as it may contain information you consider private -- e.g. names of people in photos, place names etc. This command dumps all the raw info that osxphotos knows about the photos with those particular UUIDs.

[neebah]

Here is the debug_dump

I have a hunch that this problem deals with photos shared with me that I haven't saved officially yet.

Opening database: Done; took 10.91 seconds _dbphotos['50098B3B-5442-47AA-8F8C-B0BF0270965E']: {'UTI': 'public.heic', 'UTI_original': None, 'UTI_raw': None, '_uuid': '50098B3B-5442-47AA-8F8C-B0BF0270965E', 'added_date': datetime.datetime(2021, 10, 30, 22, 56, 53, 162543), 'adjustmentFormatID': None, 'adjustmentUuid': None, 'albums': [], 'alt_master_uuid': None, 'burst': False, 'burstPickType': 0, 'burstUUID': None, 'cloudAssetGUID': None, 'cloudAvailable': None, 'cloudLibraryState': None, 'cloudLocalState': None, 'cloudStatus': None, 'cloudbatchpublishdate': None, 'cloudownerhashedpersonid': None, 'countryCode': None, 'customRenderedValue': 10, 'datastore_subtype': None, 'depth_state': 0, 'directory': '5', 'extendedDescription': None, 'favorite': 0, 'filename': '50098B3B-5442-47AA-8F8C-B0BF0270965E.heic', 'fok_import_session': None, 'hasAdjustments': 0, 'hasAlbums': 0, 'hasKeywords': 0, 'hasPersons': 0, 'has_raw': False, 'hdr': False, 'height': 4032, 'hidden': 0, 'imageDate': datetime.datetime(2021, 10, 30, 22, 54, 42, 702000, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=72000))), 'imageDate_timestamp': 657341682.702, 'imageTimeZoneOffsetSeconds': -14400, 'import_session': None, 'import_uuid': None, 'incloud': None, 'intrash': False, 'isMissing': 0, 'isreference': False, 'keywords': [], 'lastmodifieddate': None, 'lastmodifieddate_timestamp': None, 'latitude': 39.57212, 'live_photo': True, 'localAvailability': 1, 'longitude': -86.1652, 'masterFingerprint': None, 'masterUuid': None, 'modelID': None, 'momentID': 912, 'name': None, 'non_raw_master_uuid': None, 'orientation': 6, 'originalFilename': 'IMG_6467.HEIC', 'original_filesize': 1640000, 'original_height': 4032, 'original_orientation': 6, 'original_resource_choice': 0, 'original_width': 3024, 'panorama': False, 'persons': [], 'pk': 8969, 'placeIDs': None, 'placeNames': None, 'portrait': False, 'raw_data_length': None, 'raw_info': None, 'raw_is_original': False, 'raw_master_uuid': None, 'remoteAvailability': 0, 'resource_type': None, 'reverse_geolocation': None, 'saved_asset_type': 12, 'screenshot': False, 'selfie': False, 'shared': False, 'slow_mo': False, 'subtype': 2, 'time_lapse': False, 'trasheddate': None, 'trasheddate_timestamp': None, 'type': 0, 'visibility_state': 0, 'visible': True, 'width': 3024} _dbphotos['ECBEEA11-AF25-41B4-B63C-763F76B330C8']: Did not find uuid ECBEEA11-AF25-41B4-B63C-763F76B330C8 in _dbphotos _dbphotos['08DE40C8-A47A-427F-95D6-15430C6FEE80']: {'UTI': 'public.heic', 'UTI_original': None, 'UTI_raw': None, '_uuid': '08DE40C8-A47A-427F-95D6-15430C6FEE80', 'added_date': datetime.datetime(2021, 10, 30, 22, 56, 53, 162543), 'adjustmentFormatID': None, 'adjustmentUuid': None, 'albums': [], 'alt_master_uuid': None, 'burst': False, 'burstPickType': 0, 'burstUUID': None, 'cloudAssetGUID': None, 'cloudAvailable': None, 'cloudLibraryState': None, 'cloudLocalState': None, 'cloudStatus': None, 'cloudbatchpublishdate': None, 'cloudownerhashedpersonid': None, 'countryCode': None, 'customRenderedValue': 10, 'datastore_subtype': None, 'depth_state': 0, 'directory': '0', 'extendedDescription': None, 'favorite': 0, 'filename': '08DE40C8-A47A-427F-95D6-15430C6FEE80.heic', 'fok_import_session': None, 'hasAdjustments': 0, 'hasAlbums': 0, 'hasKeywords': 0, 'hasPersons': 0, 'has_raw': False, 'hdr': False, 'height': 4032, 'hidden': 0, 'imageDate': datetime.datetime(2021, 10, 30, 22, 55, 12, 935000, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=72000))), 'imageDate_timestamp': 657341712.935, 'imageTimeZoneOffsetSeconds': -14400, 'import_session': None, 'import_uuid': None, 'incloud': None, 'intrash': False, 'isMissing': 0, 'isreference': False, 'keywords': [], 'lastmodifieddate': None, 'lastmodifieddate_timestamp': None, 'latitude': 39.572095, 'live_photo': True, 'localAvailability': 1, 'longitude': -86.1651695, 'masterFingerprint': None, 'masterUuid': None, 'modelID': None, 'momentID': 912, 'name': None, 'non_raw_master_uuid': None, 'orientation': 6, 'originalFilename': 'IMG_6470.HEIC', 'original_filesize': 1693560, 'original_height': 4032, 'original_orientation': 6, 'original_resource_choice': 0, 'original_width': 3024, 'panorama': False, 'persons': [], 'pk': 8866, 'placeIDs': None, 'placeNames': None, 'portrait': False, 'raw_data_length': None, 'raw_info': None, 'raw_is_original': False, 'raw_master_uuid': None, 'remoteAvailability': 0, 'resource_type': None, 'reverse_geolocation': None, 'saved_asset_type': 12, 'screenshot': False, 'selfie': False, 'shared': False, 'slow_mo': False, 'subtype': 2, 'time_lapse': False, 'trasheddate': None, 'trasheddate_timestamp': None, 'type': 0, 'visibility_state': 0, 'visible': True, 'width': 3024} _dbphotos['093B5199-93D5-4925-9F2C-60FABED78500']: Did not find uuid 093B5199-93D5-4925-9F2C-60FABED78500 in _dbphotos

[RhetTbull]

I'll look through this later and see if I can find any hints. You can inspect the actual photos in case that gives you any clues. Try this:

osxphotos repl Then in the interactive session that opens up, type

photo = get_photo("50098B3B-5442-47AA-8F8C-B0BF0270965E")
spotlight(photo)

That will open photos and show the photo in question for that UUID.

[RhetTbull]

I have a hunch that this problem deals with photos shared with me that I haven't saved officially yet.

Those should be skipped with the --not-shared flag

[neebah]

Running the interactive sessions gave me an Attribute Error: 'NoneType' object has no attribute 'uuid'

[neebah]

importing all of my shared photos (there were only 58) seemed to fix the problem with both deleted and download-missing and without the not-shared option.

[neebah]

spoke too soon getting this error

Error exporting photo (764BEDEB-F19A-4A29-A283-3848E0455935: 6E81D59F-8A6D-4A1F-BF97-319F5FAB9507.mp4) as "location": Error Domain=NSCocoaErrorDomain Code=516 "“764BEDEB-F19A-4A29-A283-3848E0455935.medium.MP4” couldn’t be copied to “private location” because an item with the same name already exists." UserInfo={NSSourceFilePathErrorKey="private location Photo Library.photoslibrary/scopes/cloudsharing/data/1078045304/EE0E98FF-E25B-41D1-8344-DD8C254A0841/764BEDEB-F19A-4A29-A283-3848E0455935.medium.MP4, NSUserStringVariant=( Copy ), (_photoinfo_export.py: 1274)

[RhetTbull]

Running the interactive sessions gave me an Attribute Error: 'NoneType' object has no attribute 'uuid'

That means the UUID can't be found in the database which is weird as I copied it from one of your error messages.

It might be worth trying to repair the library (launch Photos while holding down Command+Option) but I'd make sure to have a backup before doing this. The repair process rebuilds the photos database.

[neebah]

Repaired the library. Had to turn off and then turn on the shared albums in preferences to get them to load.
Getting 1274 errors. All photos getting errors are photos in a shared album.

[RhetTbull]

As far as I can tell at this point it looks like osxphotos is not correctly identifying shared photos (photos in shared albums) on Monterey. I don't have a Mac capable of running Monterey so I can't test this at the moment. Could you try the interactive session again with one or two of the UUIDs that are causing errors (osxphotos repl) then

photo = get_photo("UUID here")
photo.shared

photo.shared should print True but if it prints nothing it means osxphotos doesn't recognize this as a shared photo. I'll dig into the database to see if I can figure out which of the Monterey changes could be causing this.

[neebah]

The several that I checked recognized as true.

[RhetTbull]

The several that I checked recognized as true.

So at least some of the shared photos are being recognized.

Besides the errors for what you suspect are shared photos are the rest of your photos being successfully exported?

Also, just to confirm, you're getting these errors even with the --no-shared flag?

[RhetTbull]

@all-contributors add @neebah for bug

[allcontributors[bot]]

@RhetTbull

I've put up a pull request to add @neebah! :tada:

[RhetTbull]

@all-contributors add @alandefreitas for bug

[allcontributors[bot]]

@RhetTbull

I've put up a pull request to add @alandefreitas! :tada:

[lewisthompson]

I have at least one photo that shows up with shared = False that has been shared with me (but is not from a Shared Album):

[...]
description: null
title: null
keywords: []
albums: []
persons: []
path: null
ismissing: false
hasadjustments: false
external_edit: false
favorite: false
hidden: false
latitude: //...
longitude: //...
path_edited: null
shared: false
isphoto: true
ismovie: false
uti: public.jpeg
burst: false
live_photo: true
path_live_photo: null
iscloudasset: false
incloud: null
date_modified: null
portrait: false
screenshot: false
slow_mo: false
time_lapse: false
hdr: false
selfie: true
panorama: false
has_raw: false
uti_raw: null
path_raw: null
[...]
intrash: false
[...]

Looking in Photos.app for photos taken at this exact time I see that it has the characteristic From XYZ at the top of the image.

Are there any outputs I could provide that would be helpful to debug?

[lewisthompson]

Perhaps I spoke too soon - repairing the Photos database resolved my issue.