Sonoma 14.4.1 appears to be broken with "database disk image is malformed" and "Could not decode to UTF-8 column"

[rsr-at-mindtwin]

It appears that as of upgrading to Sonoma 14.4.1 last night osxphotos is no longer happy. I have rebuilt/resynced the Photos library on the off chance that this was just Apple roaching the db during the upgrage. After repairing, the same errors persist.

Version information:

~ % osxphotos -v osxphotos, version 0.67.9 Python 3.12.2 (main, Feb 6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] macOS 14.4.1, arm64

Prologue:

Using last opened Photos library: /Users/thisuser/Pictures/Photos Library.photoslibrary Using osxphotos export database: version 9.1 located at /Users/thisuser/Library/CloudStorage/Dropbox/thisuser/0_staging/0photos/.osxphotos_export.db Processing database /Users/thisuser/Pictures/Photos Library.photoslibrary/database/photos.db Processing database /Users/thisuser/Pictures/Photos Library.photoslibrary/database/Photos.sqlite Photos database version: 6000, 9.

Then there will be a stram of hundreds of errors:

Error exporting photo (29BD1948-3024-4B49-BA4F-95B35BA475F1: IMG_2374.HEIC) as 2024-03-19 IMG_2374.HEIC: Could not decode to UTF-8 column 'photoinfo' with text '{"albums": [], "burst": false, "cloud_guid": "Ae+Rjz421ZjOadYfS/+NVe5hr6O6", "cloud_owner_hashed_id": null, "comments": [], "date": "2024-03-1 Error exporting photo (139A9E82-A08A-4AED-A17A-8895F7D325A5: IMG_0131.PNG) as 2024-01-31 IMG_0131.PNG: database disk image is malformed Error exporting photo (4196AA95-8B31-40AD-B8A4-E4B4DC34D502: IMG_1801.HEIC) as 2024-02-09 IMG_1801.HEIC: database disk image is malformed Error exporting photo (A890D367-FC42-4414-99E7-7D4E13D1DD09: IMG_1598.HEIC) as 2024-01-05 IMG_1598.HEIC: database disk image is malformed Error exporting photo (2C5FEAA5-2401-4309-A50B-23B36944273A: IMG_1607.PNG) as 2024-01-08 IMG_1607.PNG: database disk image is malformed

These are all new. Please let me know if there's any additional info I can provide.

[RhetTbull]

These look like errors in the osxphotos created export database (/Users/thisuser/Library/CloudStorage/Dropbox/thisuser/0_staging/0photos/.osxphotos_export.db) not in Photos itself. Try: osxphotos exportdb "~/Library/CloudStorage/Dropbox/thisuser/0_staging/0photos/.osxphotos_export.db" --repair

I see this appears to be on a Dropbox folder -- it's possible Dropbox has done something to the export database, which gets modified many times during an export, that caused this.

[rsr-at-mindtwin]

That does seem to have had an effect, but it's interesting --

Error exporting photo (A9C0D9FE-C77F-4B15-AFCE-DCBB5A458695: IMG_2046.HEIC) as 2024-02-25 IMG_2046.HEIC: Invalid control character at: line 1 column 449 (char 448) Error exporting photo (9723E40E-BA62-4E99-89E3-285AA639A7B9: IMG_1667.HEIC) as 2024-01-10 IMG_1667.HEIC: Expecting ',' delimiter: line 1 column 984 (char 983) Error exporting photo (2FC02E2F-AF59-49CA-9305-C642E5E808FB: IMG_2098.HEIC) as 2024-02-25 IMG_2098.HEIC: Unterminated string starting at: line 1 column 4367 (char 4366) Error exporting photo (F224B9B8-AE1C-4474-B1E2-F6C0FFA29D22: IMG_1592.HEIC) as 2024-01-05 IMG_1592.HEIC: Expecting property name enclosed in double quotes: line 1 column 4386 (char 4385) Error exporting photo (7B01068B-73E6-43C7-B442-0F030887D2AA: IMG_1853.HEIC) as 2024-02-10 IMG_1853.HEIC: Expecting ':' delimiter: line 1 column 472 (char 471) Error exporting photo (1A5754C3-51A6-4EC8-9205-1C35AD3CC09A: IMG_1768.HEIC) as 2024-02-06 IMG_1768.HEIC: Could not decode to UTF-8 column 'photoinfo' with text '{"albums": [], "burst": false, "cloud_guid": "Ab4vT1+N7mRw7RJYulb3Lii4KWbq", "cloud_owner_hashed_id": null, "comments": [], "date": "2024-02-0 Error exporting photo (DD8646D5-AE9C-4B68-8818-C633C90DA331: IMG_1926.HEIC) as 2024-02-14 IMG_1926.HEIC: Expecting ':' delimiter: line 1 column 1170 (char 1169) Error exporting photo (E957B52C-E5C1-424D-89C0-2646D1D802E1: IMG_2226.HEIC) as 2024-03-08 IMG_2226.HEIC: Could not decode to UTF-8 column 'photoinfo' with text '{"albums": [], "burst": false, "cloud_guid": "ASr6WOrpC1O4VX41goDE1X1wyoZg", "cloud_owner_hashed_id": null, "comments": [], "date": "2024-03-0 Error exporting photo (CF3AA75F-C1D9-492C-966D-E651387CD91F: IMG_2384.HEIC) as 2024-03-20 IMG_2384.HEIC: Unterminated string starting at: line 1 column 4421 (char 4420) Error exporting photo (BCD7FD8F-D71B-47DB-8AE9-E05FBC726BBA: IMG_1834.HEIC) as 2024-02-10 IMG_1834.HEIC: Invalid control character at: line 1 column 449 (char 448) Error exporting photo (2D024735-6A80-40A9-AD12-E7A014434DC8: IMG_2378.HEIC) as 2024-03-20 IMG_2378.HEIC: Unterminated string starting at: line 1 column 446 (char 445) Error exporting photo (30FD96B6-7512-4F20-B4F5-6D2712A8261E: IMG_2139.HEIC) as 2024-02-29 IMG_2139.HEIC: Unterminated string starting at: line 1 column 4224 (char 4223) Error exporting photo (7D648607-FB5C-4951-9DF0-CE95FC8C56BF: IMG_1843.HEIC) as 2024-02-10 IMG_1843.HEIC: Unterminated string starting at: line 1 column 4382 (char 4381) Error exporting photo (0896A394-71E7-41AD-ACE6-329432D1ADD5: IMG_1817.HEIC) as 2024-02-09 IMG_1817.HEIC: Expecting value: line 1 column 449 (char 448) Error exporting photo (BBD0F273-EB81-4BCC-9B1F-C4B78CB7F13B: IMG_1953.HEIC) as 2024-02-17 IMG_1953.HEIC: Invalid control character at: line 1 column 466 (char 465) Error exporting photo (0D6AE72E-7526-4983-9160-97279A5A4D32: IMG_5143.heic) as 2024-01-03 IMG_5143.heic: Unterminated string starting at: line 1 column 4082 (char 4081) Error exporting photo (253D3B34-8550-4F94-A344-8C03A24BE085: IMG_2236.HEIC) as 2024-03-09 IMG_2236.HEIC: Expecting ':' delimiter: line 1 column 463 (char 462) Error exporting photo (791BA434-8A62-4081-8743-B945C9BF0E07: IMG_1740.HEIC) as 2024-01-30 IMG_1740.HEIC: Invalid control character at: line 1 column 449 (char 448) Error exporting photo (9D432FB5-72CD-4EBA-824F-CA78426AE26B: IMG_2074.HEIC) as 2024-02-25 IMG_2074.HEIC: Expecting property name enclosed in double quotes: line 1 column 4411 (char 4410) Error exporting photo (4B12B548-EA13-4D25-948B-E5CB20ACB560: IMG_1821.HEIC) as 2024-02-10 IMG_1821.HEIC: Expecting ':' delimiter: line 1 column 4411 (char 4410) Error exporting photo (C250320F-DD98-4638-8624-B7B07071DF00: IMG_2269.HEIC) as 2024-03-16 IMG_2269.HEIC: Expecting ',' delimiter: line 1 column 969 (char 968) Error exporting photo (458515A8-2583-4607-BB47-BF625000E358: IMG_2430.HEIC) as 2024-03-25 IMG_2430.HEIC: Unterminated string starting at: line 1 column 4278 (char 4277) Error exporting photo (9470298E-EA12-446F-AB53-C22E0012565E: IMG_1719.HEIC) as 2024-01-22 IMG_1719.HEIC: Unterminated string starting at: line 1 column 4256 (char 4255)

prologue:

Using last opened Photos library: /Users/rsr/Pictures/Photos Library.photoslibrary Using osxphotos export database: version 9.1 located at /Users/rsr/Library/CloudStorage/Dropbox/thisuser/0_staging/0photos/.osxphotos_export.db

[RhetTbull]

It appears your export database is corrupt. This might work. Run the following from command line:

sqlite3 "/Users/rsr/Library/CloudStorage/Dropbox/thisuser/0_staging/0photos/.osxphotos_export.db" "UPDATE photoinfo SET photoinfo = '{}';"

Update the path to your actual path of course.

[rsr-at-mindtwin]

That did in fact fix it, thank you.