search

ostark / craft-async-queue

Async Queue Handler for Craft 3 and 4
MIT License
93 stars 7 forks source link

Job Stuck in Pending State #31

Closed josephkerkhof closed 3 years ago

josephkerkhof commented 4 years ago

Hello,

I recently upgraded from version 1.4 to 2.1.1, but I seem to be encountering a situation where jobs remain stuck at the Pending state.

I'm not seeing any error messages or anything pointing to a problem. Downgrading back to 1.4 resolves my issues.

Any help would be appreciated.

jonlongnecker commented 4 years ago

Same here - I just about blew up my computer with never ending pending jobs stacking up over and over again. Uninstalling Async Queue made everything start moving again. This was on a fresh local install with current versions of everything running on MAMP.

br-designer commented 4 years ago

Same issue. Cannot update from 1.3.1 to 2.1.1.

ostark commented 4 years ago

@br-designer Do you use MAMP as well?

ostark commented 4 years ago

@jonlongnecker @jonlongnecker @musicaljoeker try this in your .env

PHP_BINARY="/Applications/MAMP/bin/php/php{your.php.version}/bin/php"

Replace {your.php.version} with your actual version, like 7.4.2.

br-designer commented 4 years ago

@ostark yes, I use Mamp as well. I'll test out the code above today.

outline4 commented 4 years ago

Same Problem here. But it's on a live site... PHP 7.4.10
Craft CMS 3.5.5

ostark commented 4 years ago

@outline4 Always? Sometimes?

outline4 commented 4 years ago

Always!

Deactivating the plugin solves the problem.

I also downgraded to 1.4... But I am not sure if this works: Client hasn't had any orders till the downgrade...

Craft Queue seems to work fine.

ostark commented 4 years ago

When you go to your-site.com/admin/utilities/async-queue-test (requires 2.0+) What's the value for "Command Line"?

outline4 commented 4 years ago

I also downgraded to 1.4... But I am not sure if this works: Client hasn't had any orders till the downgrade...

The client just told me that it's working fine.

outline4 commented 4 years ago

When you go to your-site.com/admin/utilities/async-queue-test (requires 2.0+) What's the value for "Command Line"?

Since now I have 1.4 I can't really tell...

ul8 commented 4 years ago

similar issue here - job gets stuck indefinitely in pending state when I retry it.

Not a big problem for me however, as running the async queue test jobs will trigger it to run correctly, along with the test jobs.

command line value is "nice /usr/local/bin/php craft queue/run > /dev/null 2>&1 &" PHP version | 7.4.6 Linux 4.19.76-linuxkit PostgreSQL 11.6 Craft Pro 3.5.8 AsyncQueue | 2.1.1

tmulry commented 4 years ago

@ostark I am seeing this issue as well in 2.1.1 php 7.4 and craft 3.4.12 . For our implementation, the stuck jobs are happening when we set a delay on the queue. I made a PR with a test button that can easily reproduce the issue https://github.com/ostark/craft-async-queue/pull/37 .

Steps to reproduce(on the fork I made):

  1. clear the queue of all jobs
  2. Click the "Run Test with Delay" button on the test utility page

Expected: jobs start filling the queue and execute every 10 seconds

Actual: Jobs enter the queue every ten seconds but never move past the "Pending" state.

Some additional observations

  1. If I wait for the queue to fill with tasks and then click "Run test with Delay" or "Run Tests" it seems to kickstart the queue and the pending tasks start running.
  2. If I reduce the delay increment from 10 seconds per task to 1 second per task, then none of the tasks get stuck.
ostark commented 4 years ago

I never worked with delays. @tmulry
I think your observations (thanks for taking the time to write it down) are kind of expected with the current implementation.

Can you give a good example for delayed queue execution?

tmulry commented 4 years ago

@ostark The main usage for this client is to schedule a daily or weekly automated mailing that takes a long time to complete, so needs to run via a background task and cannot be kicked off manually.

ostark commented 4 years ago

You should use a cron for this.

tmulry commented 4 years ago

I agree this is a very cron-like setup. It has been working for us for a couple of years to use delay to schedule tasks in version 1.4 of craft-async-queue. Can you provide some insight as to what changed between 1.4 and 2.1 that makes using delays not viable with this plugin?

ostark commented 4 years ago

Honestly, I can't imagine the delay worked properly in 1.x. - here is the diff.

edhebert commented 3 years ago

Same problem here, unfortunately.

Async Queue 2.1.1 Craft 3.5.17.1 PHP 7.4.13

Disabling the plugin allows the queue to execute as expected.

edhebert commented 3 years ago

Here's a screen shot of my Async Queue Test

Screen Shot 2020-12-21 at 9 09 16 AM
ostark commented 3 years ago

@edhebert try to set the PHP_BINARY ENV var to /opt/cpanel/ea-php74/root/usr/bin/php https://github.com/ostark/craft-async-queue#configuration-optional

edhebert commented 3 years ago

That solved the issue - thanks, and happy holidays to you!

zhua89cc commented 3 years ago

@ostark I'm actually having a similar issue as well... weirdly enough only on my production environment.

image I messed with the concurrency env variable to see if that would fix it. 4 should be more than enough. We recently updated craft to 3.6 so not sure if there are breaking changes there. We also bumped our php version to 7.4 with the update as well.

zhua89cc commented 3 years ago

EDIT: I managed to resolve the issue by updating the PHP BINARY Env Variable to reference php7.4-zts under usr/bin. I decided to run the command line in async queue manually on the server and got the below error

PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20160303-zts/sodium.so' - /usr/lib/php/20160303-zts/sodium.so: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Cannot load module 'memcached' because required module 'msgpack' is not loaded in Unknown on line 0 Craft requires PHP 7.2.5 or later.

I changed the php version to 7.4 and it worked after that.

That seemed to fix it. not sure why it doesn't break on my other environments though. Thanks!