error opening osxphotos export db during export

jasonhollis commented 6 months ago

I'm not sure what the bug is but I don't think it likes --keyword "álainn" My destination is also a folder called álainn but álainn is listed as a keyword when I run osxphotos info.

osxphotos export /Volumes/Jason/álainn --exiftool --update --download-missing --finder-tag-keywords --keyword “álainn” Using last opened Photos library: /Users/jason/Pictures/Photos Library.photoslibrary Something went wrong and osxphotos encountered an error: disk I/O error Crash log written to '/Volumes/Jason/Photos/osxphotos_crash.log' Please file a bug report at https://github.com/RhetTbull/osxphotos/issues with the crash log attached.

I expected it to extract all of the photos with the keyword "álainn" into /Volumes/Jason/álainn

MacOS 14.3.1 (23D60) You have the latest version of osxphotos: 0.67.6

jasonhollis commented 6 months ago

osxphotos_crash.log

RhetTbull commented 6 months ago

This isn't due to the keyword or the unicode characters in the keyword. I created a folder named álainn and a keyword with the same and then ran your export command and it worked fine (see below). There's at least 2 problems;

You used "smart quotes" with your keyword --keyword “álainn” passed the keyword as “álainn” not as álainn, in other words, it included the quotes. I'm guessing you typed the command first in TextEdit or Notes then pasted it to Terminal. Unless explicitly disabled, MacOS will convert the regular " mark to a "smart quote" “ also known as a curly quote. The shell and thus osxphotos see this as just another character, not a quote and include it as part of the keyword. OSXPhotos will warn you if you do this in a file path but for things like keywords, it assumes you meant what you typed.
The error your getting has nothing to do with the keyword (even though the keyword argument was malformed) but looks like an error occurred during creation or migration of the export database as crash occurs in the code that is opening the export database which OSXPhotos uses for tracking which photos have been exported.

Were you exporting into a previously used export location? That is, you had run osxphotos export in the past to export photos here and now you want to export new photos using the --update? If not, the easiest thing to do is to delete the export database (There's a file .osxphotos_export.db in the folder /Volumes/Jason/álainn that you should delete). You won't see this in Finder as it's hidden so open the folder in Finder, then press shift+command+. (shift command dot) This will toggle the "view dot files" feature in Finder. Find .osxphotos_export.db and delete it. Alternatively, in the Terminal, type: rm /Volumes/Jason/álainn/.osxphotos_export.db*

If this was a previous export and you want to try preserving the export state, you could try these command:

osxphotos exportdb --repair Volumes/Jason/álainn
osxphotos exportdb --upgrade Volumes/Jason/álainn

And run the command again (this time without the smart quotes).

Alternatively, you could delete the entire export directory and re-run the original export.

I'm not sure what happened to get the database into this state though. Did osxphotos crash on you at some point prior to this? Did you cancel an export? Was this the first time running the export in this folder? Any other data you can provide would help me track down the source of the actual issue.

osxphotos export ~/Desktop/export/álainn --exiftool --update --download-missing --finder-tag-keywords --keyword álainn --verbose
osxphotos version: 0.67.6
Python version: 3.11.5 (main, Aug 25 2023, 01:57:11) [Clang 14.0.3 (clang-1403.0.22.14.1)]
Platform: macOS-13.5.1-x86_64-i386-64bit, 13.5.1
Verbose level: 1
exiftool path: /usr/local/bin/exiftool
Using last opened Photos library: /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary
Using export database /Users/rhet/Desktop/export/álainn/.osxphotos_export.db
Processing database /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary/database/photos.db
Processing database /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary/database/Photos.sqlite
Processing database.
Database version: 6000, 8.
Processing persons in photos.
Processing detected faces in photos.
Processing albums.
Processing keywords.
Processing photo details.
Processing import sessions.
Processing additional photo details.
Processing face details.
Processing photo labels.
Processing EXIF details.
Processing computed aesthetic scores.
Processing comments and likes for shared photos.
Processing moments.
Processing syndication info.
Processing shared iCloud library info
Done processing details from Photos library.
Exporting 2 photos to /Users/rhet/Desktop/export/álainn...
Exporting wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51.jpeg) (1/2)
Writing metadata with exiftool for wedding.jpg
Exported wedding.jpg to /Users/rhet/Desktop/export/álainn/wedding.jpg
Exported new file /Users/rhet/Desktop/export/álainn/wedding.jpg
Exporting edited version of wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51.jpeg)
Writing metadata with exiftool for wedding_edited.jpeg
Exported wedding.jpg to /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg
Exported new file /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg
Writing Finder tags to /Users/rhet/Desktop/export/álainn/wedding.jpg
Writing Finder tags to /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg
Skipping Finder tags for /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg: nothing to do
Skipping Finder tags for /Users/rhet/Desktop/export/álainn/wedding.jpg: nothing to do
Exporting exif1.jpg (407A44AE-30AF-4E54-8549-9DB0B1B48861.jpeg) (2/2)
Writing metadata with exiftool for exif1.jpg
Exported exif1.jpg to /Users/rhet/Desktop/export/álainn/exif1.jpg
Exported new file /Users/rhet/Desktop/export/álainn/exif1.jpg
Writing Finder tags to /Users/rhet/Desktop/export/álainn/exif1.jpg
Skipping Finder tags for /Users/rhet/Desktop/export/álainn/exif1.jpg: nothing to do
Exporting 2 photos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Processed: 2 photos, exported: 3, updated: 0, skipped: 0, updated EXIF data: 3, missing: 0, error: 0
Elapsed time: 0:00:01
Cleaning up lock files

jasonhollis commented 6 months ago

Rhet,

You’re a legend! Yea, I don’t usually submit anything and would have ignored it, but your code is so cool in doing the dump and giving me the URL. I figured, why not? All your points are accurate. I was using notes, which I use for everything in about 7 devices. The other thing that it does that drives me nuts is it converts -- to C. I was actually running a second instance in parallel to a much bigger job on the same machine, which is probably why the database issue occurred.

I have only started playing with OSXPhotos about 3 days ago, but I have not noticed any forums or anywhere obvious to get advice. Yesterday I was trying to work out the simplest way to schedule OSXPhotos to run on a routine basis using the --update switch. I realize I’m taking the easy way out but it would save me a LOT of time if you could give me your two cents. I have looked at shortcuts, Applescript and could probably leverage the crontab. The biggest issue I seem to be facing, no matter which method, is getting the correct path and environment specified. I would be very grateful if you could help me with those two items and any advice you have on which method is most optimal.

The second thought I had you might find interesting. I am leveraging a NAS for the export. I noticed in your docs the notion of using a ramdisk for the .osxphotos_export.db. I was considering a launch script that copies \Volumes\Jason\Photos.osxphotos_export.db to the ramdisk, then runs the OSXPhotos operation and then copies the db back to the NAS on completion. I’m curious about your thoughts on that approach? Not that I’m questioning your code because I think it’s way comprehensive, but I’m guessing you don’t build out the DB in an internal array because of memory use?

I have read the docs a couple of times quickly. My gut tells me you already have it in there, but you have so much functionality that I will need to dig for it for a while. I have exiftool setup. Is it possible to change the creation date or the modified date in the exported files to match the creation date from the meta data in Photos?

I’m happy to spend the time posting thoughts, questions and or answers somewhere if you point me to where that should be.

I really appreciate the work that you are doing. It’s very cool stuff.

Best regards,

Jason

Best regards,

Jason

From: Rhet Turnbull @.> Date: Thursday, 7 March 2024 at 01:00 To: RhetTbull/osxphotos @.> Cc: Jason Hollis @.>, Author @.> Subject: Re: [RhetTbull/osxphotos] Don't think it likes Irish (Issue #1435) You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification

This isn't due to the keyword or the unicode characters in the keyword. I created a folder named álainn and a keyword with the same and then ran your export command and it worked fine (see below). There's at least 2 problems;

You used "smart quotes" with your keyword --keyword “álainn” passed the keyword as “álainn” not as álainn, in other words, it included the quotes. I'm guessing you typed the command first in TextEdit or Notes then pasted it to Terminal. Unless explicitly disabled, MacOS will convert the regular " mark to a "smart quote" “ also known as a curly quote. The shell and thus osxphotos see this as just another character, not a quote and include it as part of the keyword. OSXPhotos will warn you if you do this in a file path but for things like keywords, it assumes you meant what you typed.
The error your getting has nothing to do with the keyword (even though the keyword argument was malformed) but looks like an error occurred during creation or migration of the export database as crash occurs in the code that is opening the export database which OSXPhotos uses for tracking which photos have been exported.

Were you exporting into a previously used export location? That is, you had run osxphotos export in the past to export photos here and now you want to export new photos using the --update? If not, the easiest thing to do is to delete the export database (There's a file .osxphotos_export.db in the folder /Volumes/Jason/álainn that you should delete). You won't see this in Finder as it's hidden so open the folder in Finder, then press shift+command+. (shift command dot) This will toggle the "view dot files" feature in Finder. Find .osxphotos_export.db and delete it. Alternatively, in the Terminal, type: rm /Volumes/Jason/álainn/.osxphotos_export.db*

If this was a previous export and you want to try preserving the export state, you could try these command:

osxphotos exportdb --repair Volumes/Jason/álainn

osxphotos exportdb --upgrade Volumes/Jason/álainn

And run the command again (this time without the smart quotes).

Alternatively, you could delete the entire export directory and re-run the original export.

I'm not sure what happened to get the database into this state though. Did osxphotos crash on you at some point prior to this? Did you cancel an export? Was this the first time running the export in this folder? Any other data you can provide would help me track down the source of the actual issue.

osxphotos export ~/Desktop/export/álainn --exiftool --update --download-missing --finder-tag-keywords --keyword álainn --verbose

osxphotos version: 0.67.6

Python version: 3.11.5 (main, Aug 25 2023, 01:57:11) [Clang 14.0.3 (clang-1403.0.22.14.1)]

Platform: macOS-13.5.1-x86_64-i386-64bit, 13.5.1

Verbose level: 1

exiftool path: /usr/local/bin/exiftool

Using last opened Photos library: /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary

Using export database /Users/rhet/Desktop/export/álainn/.osxphotos_export.db

Processing database /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary/database/photos.db

Processing database /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary/database/Photos.sqlite

Processing database.

Database version: 6000, 8.

Processing persons in photos.

Processing detected faces in photos.

Processing albums.

Processing keywords.

Processing photo details.

Processing import sessions.

Processing additional photo details.

Processing face details.

Processing photo labels.

Processing EXIF details.

Processing computed aesthetic scores.

Processing comments and likes for shared photos.

Processing moments.

Processing syndication info.

Processing shared iCloud library info

Done processing details from Photos library.

Exporting 2 photos to /Users/rhet/Desktop/export/álainn...

Exporting wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51.jpeg) (1/2)

Writing metadata with exiftool for wedding.jpg

Exported wedding.jpg to /Users/rhet/Desktop/export/álainn/wedding.jpg

Exported new file /Users/rhet/Desktop/export/álainn/wedding.jpg

Exporting edited version of wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51.jpeg)

Writing metadata with exiftool for wedding_edited.jpeg

Exported wedding.jpg to /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg

Exported new file /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg

Writing Finder tags to /Users/rhet/Desktop/export/álainn/wedding.jpg

Writing Finder tags to /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg

Skipping Finder tags for /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg: nothing to do

Skipping Finder tags for /Users/rhet/Desktop/export/álainn/wedding.jpg: nothing to do

Exporting exif1.jpg (407A44AE-30AF-4E54-8549-9DB0B1B48861.jpeg) (2/2)

Writing metadata with exiftool for exif1.jpg

Exported exif1.jpg to /Users/rhet/Desktop/export/álainn/exif1.jpg

Exported new file /Users/rhet/Desktop/export/álainn/exif1.jpg

Writing Finder tags to /Users/rhet/Desktop/export/álainn/exif1.jpg

Skipping Finder tags for /Users/rhet/Desktop/export/álainn/exif1.jpg: nothing to do

Exporting 2 photos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

Processed: 2 photos, exported: 3, updated: 0, skipped: 0, updated EXIF data: 3, missing: 0, error: 0

Elapsed time: 0:00:01

Cleaning up lock files

― Reply to this email directly, view it on GitHubhttps://github.com/RhetTbull/osxphotos/issues/1435#issuecomment-1980938002, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEWUHAWV7CM6LZKBB5UMD3LYW4OOPAVCNFSM6AAAAABEIPYDHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBQHEZTQMBQGI. You are receiving this because you authored the thread.Message ID: @.***>

jasonhollis commented 6 months ago

Rhet,

Sorry with fresh eyes this morning I just noticed: If you have questions, would like to show off projects created with OSXPhotos, or if you just want to say hello, please use the GitHub discussions forumhttps://github.com/RhetTbull/osxphotos/discussions or the osxphotos subreddithttps://www.reddit.com/r/osxphotos/ on Reddit. I will shove my questions up there.

Best regards,

Jason

From: Jason Hollis @.> Date: Thursday, 7 March 2024 at 09:35 To: RhetTbull/osxphotos @.>, RhetTbull/osxphotos @.> Cc: Author @.> Subject: Re: [RhetTbull/osxphotos] Don't think it likes Irish (Issue #1435) Rhet,

You’re a legend! Yea, I don’t usually submit anything and would have ignored it, but your code is so cool in doing the dump and giving me the URL. I figured, why not? All your points are accurate. I was using notes, which I use for everything in about 7 devices. The other thing that it does that drives me nuts is it converts -- to C. I was actually running a second instance in parallel to a much bigger job on the same machine, which is probably why the database issue occurred.

I have only started playing with OSXPhotos about 3 days ago, but I have not noticed any forums or anywhere obvious to get advice. Yesterday I was trying to work out the simplest way to schedule OSXPhotos to run on a routine basis using the --update switch. I realize I’m taking the easy way out but it would save me a LOT of time if you could give me your two cents. I have looked at shortcuts, Applescript and could probably leverage the crontab. The biggest issue I seem to be facing, no matter which method, is getting the correct path and environment specified. I would be very grateful if you could help me with those two items and any advice you have on which method is most optimal.

The second thought I had you might find interesting. I am leveraging a NAS for the export. I noticed in your docs the notion of using a ramdisk for the .osxphotos_export.db. I was considering a launch script that copies \Volumes\Jason\Photos.osxphotos_export.db to the ramdisk, then runs the OSXPhotos operation and then copies the db back to the NAS on completion. I’m curious about your thoughts on that approach? Not that I’m questioning your code because I think it’s way comprehensive, but I’m guessing you don’t build out the DB in an internal array because of memory use?

I have read the docs a couple of times quickly. My gut tells me you already have it in there, but you have so much functionality that I will need to dig for it for a while. I have exiftool setup. Is it possible to change the creation date or the modified date in the exported files to match the creation date from the meta data in Photos?

I’m happy to spend the time posting thoughts, questions and or answers somewhere if you point me to where that should be.

I really appreciate the work that you are doing. It’s very cool stuff.

Best regards,

Jason

Best regards,

Jason

From: Rhet Turnbull @.> Date: Thursday, 7 March 2024 at 01:00 To: RhetTbull/osxphotos @.> Cc: Jason Hollis @.>, Author @.> Subject: Re: [RhetTbull/osxphotos] Don't think it likes Irish (Issue #1435) You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification

This isn't due to the keyword or the unicode characters in the keyword. I created a folder named álainn and a keyword with the same and then ran your export command and it worked fine (see below). There's at least 2 problems;

You used "smart quotes" with your keyword --keyword “álainn” passed the keyword as “álainn” not as álainn, in other words, it included the quotes. I'm guessing you typed the command first in TextEdit or Notes then pasted it to Terminal. Unless explicitly disabled, MacOS will convert the regular " mark to a "smart quote" “ also known as a curly quote. The shell and thus osxphotos see this as just another character, not a quote and include it as part of the keyword. OSXPhotos will warn you if you do this in a file path but for things like keywords, it assumes you meant what you typed.
The error your getting has nothing to do with the keyword (even though the keyword argument was malformed) but looks like an error occurred during creation or migration of the export database as crash occurs in the code that is opening the export database which OSXPhotos uses for tracking which photos have been exported.

Were you exporting into a previously used export location? That is, you had run osxphotos export in the past to export photos here and now you want to export new photos using the --update? If not, the easiest thing to do is to delete the export database (There's a file .osxphotos_export.db in the folder /Volumes/Jason/álainn that you should delete). You won't see this in Finder as it's hidden so open the folder in Finder, then press shift+command+. (shift command dot) This will toggle the "view dot files" feature in Finder. Find .osxphotos_export.db and delete it. Alternatively, in the Terminal, type: rm /Volumes/Jason/álainn/.osxphotos_export.db*

If this was a previous export and you want to try preserving the export state, you could try these command:

osxphotos exportdb --repair Volumes/Jason/álainn

osxphotos exportdb --upgrade Volumes/Jason/álainn

And run the command again (this time without the smart quotes).

Alternatively, you could delete the entire export directory and re-run the original export.

I'm not sure what happened to get the database into this state though. Did osxphotos crash on you at some point prior to this? Did you cancel an export? Was this the first time running the export in this folder? Any other data you can provide would help me track down the source of the actual issue.

osxphotos export ~/Desktop/export/álainn --exiftool --update --download-missing --finder-tag-keywords --keyword álainn --verbose

osxphotos version: 0.67.6

Python version: 3.11.5 (main, Aug 25 2023, 01:57:11) [Clang 14.0.3 (clang-1403.0.22.14.1)]

Platform: macOS-13.5.1-x86_64-i386-64bit, 13.5.1

Verbose level: 1

exiftool path: /usr/local/bin/exiftool

Using last opened Photos library: /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary

Using export database /Users/rhet/Desktop/export/álainn/.osxphotos_export.db

Processing database /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary/database/photos.db

Processing database /Users/rhet/Pictures/Test-13.0.0 copy.photoslibrary/database/Photos.sqlite

Processing database.

Database version: 6000, 8.

Processing persons in photos.

Processing detected faces in photos.

Processing albums.

Processing keywords.

Processing photo details.

Processing import sessions.

Processing additional photo details.

Processing face details.

Processing photo labels.

Processing EXIF details.

Processing computed aesthetic scores.

Processing comments and likes for shared photos.

Processing moments.

Processing syndication info.

Processing shared iCloud library info

Done processing details from Photos library.

Exporting 2 photos to /Users/rhet/Desktop/export/álainn...

Exporting wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51.jpeg) (1/2)

Writing metadata with exiftool for wedding.jpg

Exported wedding.jpg to /Users/rhet/Desktop/export/álainn/wedding.jpg

Exported new file /Users/rhet/Desktop/export/álainn/wedding.jpg

Exporting edited version of wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51.jpeg)

Writing metadata with exiftool for wedding_edited.jpeg

Exported wedding.jpg to /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg

Exported new file /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg

Writing Finder tags to /Users/rhet/Desktop/export/álainn/wedding.jpg

Writing Finder tags to /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg

Skipping Finder tags for /Users/rhet/Desktop/export/álainn/wedding_edited.jpeg: nothing to do

Skipping Finder tags for /Users/rhet/Desktop/export/álainn/wedding.jpg: nothing to do

Exporting exif1.jpg (407A44AE-30AF-4E54-8549-9DB0B1B48861.jpeg) (2/2)

Writing metadata with exiftool for exif1.jpg

Exported exif1.jpg to /Users/rhet/Desktop/export/álainn/exif1.jpg

Exported new file /Users/rhet/Desktop/export/álainn/exif1.jpg

Writing Finder tags to /Users/rhet/Desktop/export/álainn/exif1.jpg

Skipping Finder tags for /Users/rhet/Desktop/export/álainn/exif1.jpg: nothing to do

Exporting 2 photos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

Processed: 2 photos, exported: 3, updated: 0, skipped: 0, updated EXIF data: 3, missing: 0, error: 0

Elapsed time: 0:00:01

Cleaning up lock files

― Reply to this email directly, view it on GitHubhttps://github.com/RhetTbull/osxphotos/issues/1435#issuecomment-1980938002, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEWUHAWV7CM6LZKBB5UMD3LYW4OOPAVCNFSM6AAAAABEIPYDHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBQHEZTQMBQGI. You are receiving this because you authored the thread.Message ID: @.***>

oPromessa commented 6 months ago

Hey there. On scheduling osxphotos to run see discussion #1080

oPromessa commented 6 months ago

Me again.

On your topic

The second thought I had you might find interesting. I am leveraging a NAS for the export. I noticed in your docs the notion of using a ramdisk for the .osxphotos_export.db. I was considering a launch script that copies \Volumes\Jason\Photos.osxphotos_export.db to the ramdisk, then runs the OSXPhotos operation and then copies the db back to the NAS on completion.

I'll let Rhet comment further but I believe that's what osxphotos already does.

I also export to NAS. Besides using --ramdb I also use --exportdb option and place the exportdb in the local Mac disk and not on the NAS. This was more relevant prior to --ramdb but still avoids some NAS traffic and the beginning and end of the run.

You must however keep track of the location of your exportdb.

RhetTbull commented 6 months ago

I was actually running a second instance in parallel to a much bigger job on the same machine, which is probably why the database issue occurred.

The database should be safe for access in parallel but this isn't an explicitly supported use case. I've added code to catch this error and attempt to repair the database but that's not been released yet.

Yesterday I was trying to work out the simplest way to schedule OSXPhotos to run on a routine basis using the --update switch.

See this discussion where @rajscode outlines the steps he uses to do this

I am building a companion app that has a GUI and will allow you to schedule periodic exports. I'll announce it on the Discussions page when ready for testing. My plan is for this to be a paid app to support osxphotos development.

I am leveraging a NAS for the export. I noticed in your docs the notion of using a ramdisk for the .osxphotos_export.db.

If using a NAS, you should use the --ramdb which copies the export database to RAM during the export then back to the NAS when done. You can also use --exportdb with or without --ramdb to specify an alternate path to the database (Eg on an internal disk). If you do this you should move the .osxphotos_export.db to the new location first.

but I'm guessing you don't build out the DB in an internal array because of memory use?

Not exactly. OSXPhotos is actually a memory hog because it loads the whole Photos database into memory. It's just that SQLite is so much easier to work with as a persistent format. There's also a whole set of tools for working with the export database under osxphotos exportdb and it can be opened in any SQLite editor. But it has very poor performance on a NAS.

Is it possible to change the creation date or the modified date in the exported files to match the creation date from the meta data in Photos?

You can set modification time to match the time in Photos with --touch-file. It is not (easily) possible to change creation date on APFS which is the default for MacOS. Your NAS might be able to but that's not the primary export target.

oPromessa commented 6 months ago

On your topic

I have exiftool setup. Is it possible to change the creation date or the modified date in the exported files to match the creation date from the meta data in Photos?

That's the purpose of the --exiftool and --touch options. Give --touch option a go.

jasonhollis commented 6 months ago

Mate,

I figured that out! The code is fast. I have a 10 gigabit Thunderbolt interface on the Mac and my Qnap has two 25 gigabit links that are bonded. I have about 40 terabytes of pure PCI SSD. I just used the --ramdb option with 9491 objects in the Photos library (that I think are all local at this point) and it looks like about 34 min of runtime. The Photos DB is about 160 gigabytes.

Best regards,

Jason

From: oPromessa @.> Date: Thursday, 7 March 2024 at 10:41 To: RhetTbull/osxphotos @.> Cc: Jason Hollis @.>, Author @.> Subject: Re: [RhetTbull/osxphotos] error opening osxphotos export db during export (Issue #1435) You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification

Me again.

On your topic

The second thought I had you might find interesting. I am leveraging a NAS for the export. I noticed in your docs the notion of using a ramdisk for the .osxphotos_export.db. I was considering a launch script that copies \Volumes\Jason\Photos.osxphotos_export.db to the ramdisk, then runs the OSXPhotos operation and then copies the db back to the NAS on completion.

— Reply to this email directly, view it on GitHubhttps://github.com/RhetTbull/osxphotos/issues/1435#issuecomment-1982049343, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEWUHAVWJZLEJ3GCE52XUD3YW6SQ3AVCNFSM6AAAAABEIPYDHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBSGA2DSMZUGM. You are receiving this because you authored the thread.Message ID: @.***>

oPromessa commented 6 months ago

On your topic

OSXPhotos to run on a routine basis using the --update switch

See also --cleanup option and comments on #1180 in case you want to delete exported files you've deleted from the Photos app.

jasonhollis commented 6 months ago

I’m onto the --touch option now. I appreciate all the help. Happy to pay for the GUI to contribute to your development and happy to test it. I will have a look at the forums now and see if I can help anyone else. As per my last email I have some pretty crazy kit to test stuff with … my current laptops are M1Max and M2Max so I have no issue with something being a memory hog. I would love options to use it all. I have 128 gigs in the M2Max and I have never touched the sides of it even editing video.

Best regards,

Jason

From: oPromessa @.> Date: Thursday, 7 March 2024 at 10:45 To: RhetTbull/osxphotos @.> Cc: Jason Hollis @.>, Author @.> Subject: Re: [RhetTbull/osxphotos] error opening osxphotos export db during export (Issue #1435) You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification

On your topic

I have exiftool setup. Is it possible to change the creation date or the modified date in the exported files to match the creation date from the meta data in Photos?

That's the purpose of the --exiftool and --touch options. Give --touch option ago.

— Reply to this email directly, view it on GitHubhttps://github.com/RhetTbull/osxphotos/issues/1435#issuecomment-1982054792, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEWUHASEH7QOQ74FWJ75GALYW6TANAVCNFSM6AAAAABEIPYDHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBSGA2TINZZGI. You are receiving this because you authored the thread.Message ID: @.***>

jasonhollis commented 6 months ago

Yes sir I did already read that bit

Best regards,

Jason

From: oPromessa @.> Date: Thursday, 7 March 2024 at 10:49 To: RhetTbull/osxphotos @.> Cc: Jason Hollis @.>, Author @.> Subject: Re: [RhetTbull/osxphotos] error opening osxphotos export db during export (Issue #1435) You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification

On your topic

OSXPhotos to run on a routine basis using the --update switch

See also --cleanup option and comments on #1180https://github.com/RhetTbull/osxphotos/discussions/1180 in case you want to delete exported files you've deleted from the Photos app.

— Reply to this email directly, view it on GitHubhttps://github.com/RhetTbull/osxphotos/issues/1435#issuecomment-1982059368, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEWUHAVO3YTGKZLXKKFC773YW6TRXAVCNFSM6AAAAABEIPYDHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBSGA2TSMZWHA. You are receiving this because you authored the thread.Message ID: @.***>

jasonhollis commented 6 months ago

Rhet,

I tried to send you a screenshot but it bounced due to filesize. I have not researched this error yet. I’m sure it’s do to the switch combinations I’m using but FYI:

@.*** ~ % osxphotos export /Volumes/jasonfast/Photos --exiftool --update --ramdb --download-missing --finder-tag-keywords --directory "{folder_album}"

Using last opened Photos library: /Users/jason/Pictures/Photos Library.photoslibrary

2024-03-07 10:30:13,550 - root - WARNING - photosdb.py - 157 - WARNING: This module has only been tested with macOS versions [10.12, 10.13, 10.14, 10.15, 10.16, 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 12.0, 12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 12.7, 13.0, 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 14.0, 14.1]: you have Darwin, OS version: 14.3

Processing database /Users/jason/Pictures/Photos

Library.photoslibrary/database/photos.db

Processing database /Users/jason/Pictures/Photos

Library.photoslibrary/database/Photos.sqlite

Processing database.

Database version: 6000, 9.

Processing persons in photos.

Processing detected faces in photos.

Processing albums.

Processing keywords.

Processing photo details.

Processing import sessions.

Processing additional photo details.

Processing face details.

Processing photo labels.

Processing EXIF details.

Processing computed aesthetic scores.

Processing comments and likes for shared photos.

Processing moments.

Processing syndication info.

Processing shared iCloud library info

Done processing details from Photos library.

Exporting 9491 photos to /Volumes/jasonfast/Photos...

Exporting 9491 photos ━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5% 0:37:45❌️ exiftool error for file /Volumes/jasonfast/Photos/_/RenderedImage (6).HEIC: