No jobs are run in production, dev very slow

[howells]

Hi!

I'm struggling to get anywhere with this plugin unfortunately. While I can get the jobs to create, none of them are actually run even though all other jobs run as normal.

I reduced the bulk job size down to 1 but all of them remain at 0% progress.

When viewing an individual entry, the element relations field never seems to update: I just see a spinner.

Development does run, but extremely slowly.

Any ideas on how I can debug?

[internetztube]

What does your setup look like on production? PHP version, Database version, Plugins, ...

[howells]

@internetztube hey sorry for the delay: here is a grab of the system report: https://cln.sh/Ked63C

The CMS is hosted on Fortrabbit's Pro server, using a dedicated worker to handle the queue which has 1GB dedicated to it (all other jobs work fine).

Relation updates after saving an element work quickly, but Refresh Cache jobs often just don't start or just fail after 300seconds

This is using a batch size of 10, but reducing this to 1 doesn't help.

Any ideas really appreciated!

[howells]

@internetztube oh I've also just noticed that a job is created after save for any kind of element, irregardless of whether it has an element relations. This site is particularly massive (2.2m elements!) so this means there'll be lots of unnecessary jobs being created.

[howells]

@internetztube something else that might cause the issues, the categories in question are linked to entries via matrix fields

[internetztube]

@howells Could you maybe provide me with an database dump + project files? support@internetztube.net

[howells]

@internetztube I emailed you it, thanks

[internetztube]

@howells Got the email. Thanks! Unfortunately, the issue is not easy to fix. It will take some time.

[gbowne-quickbase]

Let me know if you need another set of eyes on it. We had a slowdown in our system as well when we pushed it to production and disabled the plugin for the short term

[internetztube]

Thanks for the database dump! Helped a lot. Performance should be much better with 1.3.4. Although more queuing tasks are now created, duplicate relation generation is now minimized. @howells @gbowne-quickbase

[gbowne-quickbase]

I'm testing locally and everything seems to be running a lot more quickly!

[gbowne-quickbase]

This was the only oddity I saw. It took a few hours to run the nearly 4k items, but my site is huge. But before this release it wasn't hitting 5% in a few hours.

[internetztube]

@gbowne-quickbase The problem with the buggy numbers in the utilities should also be solved.

[gbowne-quickbase]

The load on my database is significantly lower during the whole process too. Congrats on your big fix

[internetztube]

If you encounter similar performance problems, feel free to reopen this issue.

[howells]

@internetztube sorry for the delay, this is working much faster now, thanks!

[howells]

@internetztube however, given the jobs are kicked off on save of any element, with lots of elements being saved every minute (new users, saved Verbb wishlist items, etc.) it kills the server. Ideally only those elements that have an element relations field associated will kick off the job, if that makes sense?

[gbowne-quickbase]

@howells how long is your cache set to? I have about 3900 assets, and kicked off the complete refresh early in the morning when the servers were quiet, and it finished in 3-4h, and I set my cache to 2 months, and haven't seen any jobs kick off on save