While the background sync system for Sunset is great for periodic updates, it's less great for trying to publish my photos in a reasonable amount of time, because once a folder syncs and finds nothing, it won't check again for several hours. When I built it, I was actually going through a bit of a photography "dry spell", so I mostly used it for bulk imports of my older work that was already published in the old gallery, and that affected my design decisions. But I'm finding it to be rather limiting now. Unfortunately, both the folder-scanning and image-processing systems are fairly slow and resource-intensive (which is why they're background processes that run as a cron job, and none of this happens upon initial import/save), so speeding it up is more complicated than just running it more often.
I had a specific idea when opening this issue, which I will implement connected to it, but it's not the whole solution. So, the problems that need solving:
When I've uploaded new images to a specific folder, and its next sync is scheduled for too far into the future, I want to be able to bump that folder to the top of the queue.
I want new images to be processed more quickly after a new import, without having to manually run the command to do it, because the whole point is that this is supposed to be a touchless import system.
I want to have more control over the sync rate for specific folders, because there are too many to only sync one per cron execution (doing that with the current configuration, each folder would only sync once every 35 hours at the most), but trying to scan all of them is too slow and resource-intensive to justify running more than once per day. And while some folders are likely to be actively updated at any time, some are unlikely to get new updates at all, but I don't want to completely disable them because, again, this is supposed to be a touchless system - if I do update images in those folders, I want them to be processed eventually, they're just unlikely to be time-sensitive.
Current implementation ideas:
Add a "sync now" setting for a folder that overrides the normal re-sync delay. Can be a boolean field for now, but it could also be handled by the resync time option system described below.
Add a per-folder resync time option of some sort. Not quite sure how to implement this, or how granular it needs to be; assuming a default of 24 hours, I would also want options for several times a day (like once every 2-4 hours), and once per week, so I could probably get by with a "sync speed" setting with options of "slow", "standard", and "fast" (or something to that effect). The most efficient option would probably be to add a "next sync" datetime field, which is set during sync based on the sync speed setting.
Switch the folder sync code to only check one folder per maintenance command execution.
Adjust the cron job schedule to run every 5 minutes instead of every 15 minutes.
While the background sync system for Sunset is great for periodic updates, it's less great for trying to publish my photos in a reasonable amount of time, because once a folder syncs and finds nothing, it won't check again for several hours. When I built it, I was actually going through a bit of a photography "dry spell", so I mostly used it for bulk imports of my older work that was already published in the old gallery, and that affected my design decisions. But I'm finding it to be rather limiting now. Unfortunately, both the folder-scanning and image-processing systems are fairly slow and resource-intensive (which is why they're background processes that run as a cron job, and none of this happens upon initial import/save), so speeding it up is more complicated than just running it more often.
I had a specific idea when opening this issue, which I will implement connected to it, but it's not the whole solution. So, the problems that need solving:
Current implementation ideas: