search

nextcloud / previewgenerator

Nextcloud app to do preview generation in the background.
https://apps.nextcloud.com/apps/previewgenerator
GNU Affero General Public License v3.0
449 stars 56 forks source link

after nextcloud restore, preview:pre-generate runs "all the time" #375

Closed fschiefenhoevel closed 1 year ago

fschiefenhoevel commented 1 year ago

Hi!

I've been using the previewgenerator app for several years now, but this seems odd: Yesterday evening, I had to restore from a nc backup after a SD Card crash. Why is preview:pre-generate running all the time since then and reading and writing to disk (both / and nc-datadir)? The filelist in /data/appdata_instanceid/preview doesn't seem to change - so as far as I can tell - no new previews are generated. Also, in the frontend, thumbnails seem to be popping up as quickly as before.

What is previewgenerator doing? Checking whether the existing previews are "correct"?

st3iny commented 1 year ago

Did you follow the guide at https://docs.nextcloud.com/server/latest/admin_manual/maintenance/restore.html?

Especially the following step is important and could be causing this: When you have completed your restoration, also make sure to run the maintenance:data-fingerprint command afterwards, to ensure your sync clients can recover from the restored backup.

Please clarify what you mean by "running all the time". Does it take longer than before? The frequency of running this command is determined by you through the cron job.

fschiefenhoevel commented 1 year ago

Hi!

thanks for your answer!

I followed the guide and have run maintenance:fingerprint after restoring. And what I mean by "all the time" is: the process preview:pre-generate is running continuously for several hours now, which it didn't do before. I have created a crontab to run it every 10 minutes (same as before). As far as I can tell, no new previews are generated though (du -hs show no change neither does ls -l | wc -l).

EDIT: sudo iotop and sudo iostat -m shows that writing does occur (on the sd card carrying the OS and the hdd having the nc_datadir), but as I mentioned, the preview directory does not seem to change in size or number of files

st3iny commented 1 year ago

My guess is that previewgenerator assumes that all files changed due to the restoration and enqueued each file again.

You could check the preview generation queue by running the following (My)SQL query: select count(*) from oc_preview_generation;

Files will be skipped if their previews already exist. Hence you don't see any new files being created on the disk.

You could kill the previewgenerator cron process and clear the whole oc_preview_generation table if this bothers you. Otherwise, you could let it run through and things should resume to normal once the queue has been processed.

fschiefenhoevel commented 1 year ago

...thanks! you are probably right. The Query shows that (as of now) >12000 files are queued. I'll let it finish. Maybe I ran the maintenance:data-fingerprint command too late or I messed up the initial config.php (didn't copy in the beginning, so a wrong instanceid was entered there).

EDIT: the size of the preview folder slightly increased since the process started hours ago. The number of directories hasn't changed though.

st3iny commented 1 year ago

Alright. I'm glad that I was able to help.

I'm closing the issue as it has been resolved.