search

nextcloud / previewgenerator

Nextcloud app to do preview generation in the background.
https://apps.nextcloud.com/apps/previewgenerator
GNU Affero General Public License v3.0
455 stars 57 forks source link

Limit number of files worked out by preview:generate-all in a row #271

Closed dgedgedge closed 2 years ago

dgedgedge commented 2 years ago

HI, I do have a lot of image files. So preview:generate-all will run for days, but I need also to backup nextcloud and the database. This implies that sometime the database is unavailable.

Would it be possible to have a preview:generate-all parametter to ask it to only process X files from index Y file to be processed and then return the index of the last processed file. The next call would process the next X files.....

The command could either store the index of the last file processed, or print it to be store by admin somewhere, and Admin would give it back to preview:generate-all at the next call.

This would also help running the command with/within the command /var/www/nextcloud/cron.php

Tanks in advance.

st3iny commented 2 years ago

That is currently not possible. You could split the initial generation into multiple paths.

Example

Assume the following folder structure for the user admin:

You could run the following commands seperately:

Of course, this won't work if all or almost all images are in the same directory.

dgedgedge commented 2 years ago

Humm yes I could proceed like that, but that means that we do the job of file search 2 times. The Idea would be to count in a file / in a global variable the previewed files (parseFile) and to stop rocessing files when the index did count up to the value. I do not know yet how to integrate correctly to nextcloud yet, and cannot propose for a patch. I hope you will have some time for that. Thans in advance.

dgedgedge commented 2 years ago

By the way is it an issue if I run "occ preview:generate-all" and send a kill, in order to stop de preview generation ? Will the first files preview be generated ?

st3iny commented 2 years ago

By the way is it an issue if I run "occ preview:generate-all" and send a kill, in order to stop de preview generation ? Will the first files preview be generated ?

Yes, all previews up to this point will be saved. If you run the command again it will process those files again but once it detects that previews are already existing it will just skip them. So in theory you could run the full command multiple times and quit whenever you need to do some maintenance.

Although it looks like all files are processed again it doesn't actually "overwrite" previews of processed files.