Open dokeraj opened 3 years ago
'generate-all' puts a pretty high load on the disk I/O and runs a reasonable time as it rescans all folders. I'd not recommend running it periodically via the standard cron, maybe once in a night.
However, I also have the feeling that 'pre-generate' has an issue and is not doing what it should as I often see missing previews.
[I'm on version Nextcloud 22.1.0]
I also have issues, My picture overview used to load pretty quickly, but it stopped being fast recently, and indeed I noticed when I do:
docker exec -u www-data nextcloud_app php occ preview:pre-generate
I get:
Command "preview:pre-generate" is not defined.
Did you mean one of these?
notification:generate
preview:repair
preview:reset-rendered-texts
When I do:
docker exec -u www-data nextcloud_app php occ preview:generate-all -vvv
I get:
In Application.php line 676:
[Symfony\Component\Console\Exception\CommandNotFoundException]
Command "preview:generate-all" is not defined.
Did you mean one of these?
notification:generate
preview:repair
preview:reset-rendered-texts
Exception trace:
at /var/www/html/3rdparty/symfony/console/Application.php:676
Symfony\Component\Console\Application->find() at /var/www/html/3rdparty/symfony/console/Application.php:237
Symfony\Component\Console\Application->doRun() at /var/www/html/3rdparty/symfony/console/Application.php:149
Symfony\Component\Console\Application->run() at /var/www/html/lib/private/Console/Application.php:209
OC\Console\Application->run() at /var/www/html/console.php:99
require_once() at /var/www/html/occ:11
So how does one generate preview these days?
@freekvh check if preview generator is still enabled on your instance. It might have been disabled during an update.
@freekvh check if preview generator is still enabled on your instance. It might have been disabled during an update.
Ah yes, I have to "enable untested app", my apologies, I didn't realize. I works now, preview:pre-generate is finished in seconds, preview:generate-all -vvv is now running and taking a long time. Do you know if preview:pre-generate is still the best thing to run?
I guess preview:generate-all regenerates everything, not advisable on a regular basis I'd say.
@freekvh check if preview generator is still enabled on your instance. It might have been disabled during an update.
Ah yes, I have to "enable untested app", my apologies, I didn't realize. I works now, preview:pre-generate is finished in seconds, preview:generate-all -vvv is now running and taking a long time. Do you know if preview:pre-generate is still the best thing to run?
I guess preview:generate-all regenerates everything, not advisable on a regular basis I'd say.
@freekvh yes, following the readme recommendations, it is not suggested to run the generate-all command regularly, only the pre-generate command should be scheduled.
@freekvh yes, following the readme recommendations, it is not suggested to run the generate-all command regularly, only the pre-generate command should be scheduled.
That's the plan, but running the preview:pre-generate via cron every 30 minutes and I'm browsing into a folder with plenty new pictures a day after they're synced into, I can watch the previews being generated one-by-one at this time. So pre-generate does not appear to work.
@freekvh yes, following the readme recommendations, it is not suggested to run the generate-all command regularly, only the pre-generate command should be scheduled.
That's the plan, but running the preview:pre-generate via cron every 30 minutes and I'm browsing into a folder with plenty new pictures a day after they're synced into, I can watch the previews being generated one-by-one at this time. So pre-generate does not appear to work.
Yes, I believe I am experiencing the same thing.
"pre-generate" does absolutely nothing at all with no error output However "generate-all" creates previews perfectly, but a lot of resources are used depending on the directory/file size
Strangely the command works when I issue it from the command line:
sudo -u www-data /usr/bin/php -f /var/www/nextcloud/occ preview:pre-generate
This is my crontab for the www-data user:
root@nextcloud:~# crontab -u www-data -l
*/5 * * * * /usr/bin/php -f /var/www/nextcloud/cron.php > /dev/null 2>&1
0 1 * * * /usr/bin/php -f /var/www/nextcloud/occ files:scan --all > /dev/null 2>&1
*/15 * * * * /usr/bin/php -f /var/www/nextcloud/occ files:scan --unscanned --all > /dev/null 2>&1
*/15 * * * * /usr/bin/php -f /var/www/nextcloud/occ preview:pre-generate
*/30 * * * * /usr/bin/php -f /var/www/nextcloud/occ fulltextsearch:index -q > /dev/null 2>&1
*/15 * * * * /usr/bin/php -f /var/www/nextcloud/occ face:background_job -t 870 > /dev/null 2>&1
0 3 * * * /usr/bin/php -f /var/www/nextcloud/occ maps:scan-photos
All commands from the are running fine except "occ preview:pre-generate".
I don't understand why it runs from the command line but not from the cron job.
Same for me.
./occ preview:pre-generate -vvv
looks to do nothing.
How to check if it's working ?
Same case:
I wonder if listener is doing its work...
Same for me.
./occ preview:pre-generate -vvv
looks to do nothing. How to check if it's working ?
Upload some image files to your server and run the command. It should generate previews for new and changed files. It will also log all processed files when -vvv
is enabled.
Upload some image files to your server and run the command. It should generate previews for new and changed files. It will also log all processed files when
-vvv
is enabled.
This is the way I did the test:
$qb->select('*') ->from('preview_generation') ->orderBy('id') ->setMaxResults(1000); $cursor = $qb->execute(); $rows = $cursor->fetchAll(); $cursor->closeCursor(); if ($this->output->getVerbosity() > OutputInterface::VERBOSITY_VERBOSE) { $this->output->writeln('preview_generation running.'); } if ($rows === []) { if ($this->output->getVerbosity() > OutputInterface::VERBOSITY_VERBOSE) { $this->output->writeln('preview_generation empty.'); } break; }
After a lot of test without getting the system to work as designed (probably because I do something wrong), I tried to find a solution that apparently works, although it is probably no more than a temporary fix. Let me explain:
protected function execute(InputInterface $input, OutputInterface $output): int { // maher 20230308 init $this->sec_ini=time(); // maher 20230308 end if ($this->encryptionManager->isEnabled()) { foreach ($nodes as $node) { // maher 20230308 init $sec_cur=time(); $sec_delta=$sec_cur-$this->sec_ini; if ($sec_delta > 500) { return; } // maher 20230308 end if ($node instanceof Folder) {
My advice if you want to try:
your nextcloud data folder
/appdataxxxxx/preview |tail -1: note the number of preview files obtained.I have to insist. This has to be just a temporary solution for those, like me, who can't get the system to work as designed (just in case this is working for you).
upload 2 image files to the server (via syncthing)
Externally added files are not picked up by pre-generate. It only handles file changes from within Nextcloud (Client, Web, Android, etc.).
If you often do external changes, you should run files:scan regularly so that Nextcloud properly picks up the changes.
upload 2 image files to the server (via syncthing)
Externally added files are not picked up by pre-generate. It only handles file changes from within Nextcloud (Client, Web, Android, etc.).
If you often do external changes, you should run files:scan regularly so that Nextcloud properly picks up the changes.
This is my cron. Do you think it's enough?
/5 php -f /var/www/nextcloud/cron.php > /dev/null 2>&1
0 10 /usr/bin/php -f /var/www/nextcloud/occ files:scan --all > /dev/null 2>&1
/19 /usr/bin/php -f /var/www/nextcloud/occ files:scan myuser
--unscanned > /dev/null 2>&1
/37 /usr/bin/php -f /var/www/nextcloud/occ preview:pre-generate
New test:
oc_preview_generation
filled (OK)oc_preview_generation
filled (OK)oc_preview_generation
filled (OK)oc_preview_generation
not filled (NOK)My conclusion: everything works as expected when using nextcloud, but this is not the case when using syncthing, even sending the same file to the same folder in Nextcloud server. Why? I don't know, and I would like to know.
files:scan
is supposed to help to solve this, as @st3iny mentioned? If so, what should be the procedure?
I tried this command in docker's container and it starts generating previews
sudo -u www-data php occ preview:generate-all -vvv
The listener, which must get triggered for pre-generate to have anything to process, is only triggered for write events within Nextcloud Server. A scan isn't sufficient.
This just sounds like #218 (unless this is occurring for non-externally added files).
Hi I have the nextcloud 21.0.0.18,
When I use the command
occ preview:pre-generate
, there is no log and it takes just a few seconds. But I can see that the images were not processed. In contrast - when I do theocc preview:generate-all
command, then, I get the log and it takes longer, but it works as expected.It says that I should use
pre-generate
in regular time intervals, but I can see that thegenerate-all
also is just searching for those images that have not been processed before. So is it ok if I just usegenerate-all
in a cronjob, instead of the other command?Am I missing something?
Thanks.