search

symbiote / silverstripe-queuedjobs

A module that provides interfaces for scheduling jobs for certain times.
BSD 3-Clause "New" or "Revised" License
57 stars 73 forks source link

Queue keeps adding duplicate jobs and long-running php processes #356

Closed muppsy007 closed 3 years ago

muppsy007 commented 3 years ago

Module version 3.1 on Silverstripe version 3.7. Doorman queue.

We had queuedjobs running on the Silverstripe 3.1 version of this site just fine. But after a recent upgrade to Silvertstripe 3.7 so we can use PHP 7.4, we find that the same queued jobs often have duplicate jobs created per iteration. Anywhere between one and five duplicate jobs per iteration, usually separated by a second or two. Even if we delete all but one of the duplicates, more duplicates quickly come back in subsequent iterations.

Additional to this (possibly a side effect/cause) is that we find a large number of php processes running and never closing. Usually after paused/broken jobs. We observed this on a test environment to the point that the queuedjobs actually spins up enough of these processes and they stick around for so long, that the server actually crashes. I believe this issue might be a known issue, but it doesn't look like this fix made it onto the 3.x version of this module.

Not sure if the two are related. Or whether not using Doorman will resolve all of this. Do we need a queueRunner at all or can the module work as it originally did under it's own steam? Would Gearman be a better choice?

emteknetnz commented 3 years ago

Unfortunately, SilverStripe 3 has entered limited support in June 2018. This means we'll only be fixing critical bugs and security issues for SilverStripe 3 going forward.

You can read the SilverStripe Roadmap for more information on our support commitments.

muppsy007 commented 3 years ago

The goto line for when the effort of reading or discussion is just too hard. @emteknetnz I asked two questions that are completely independent of version. And now this is closed so it loses visibility to anyone who might have generic advice (and 5 mins to spare giving it).

Christ the laziness of this community puts me off the entire ecosystem sometimes.

emteknetnz commented 3 years ago

The closed status is to indicate that there's no intention for the maintainers of Silverstripe to fix this bug due to it being on a version that is now only in limited support.

Closed issues are still fully visible and to everyone and people are free to have discussions on them.