vend / php-resque

PHP port of resque (Workers and Queueing)
MIT License
32 stars 9 forks source link

Be more fault tolerant. #8

Closed wotek closed 9 years ago

wotek commented 9 years ago

Lets say you have just one worker running like:

// Pulling properly configured resque from DI container
$resque = $container->get('resque');
$resque->setLogger($container->get('logger'));
$worker = new Worker($resque, array('default'));
$worker->work();

workers starts, everything looks awesome so far.

Now. Someone makes lets say typo and does:

bin/resque enqueue default 'Not\Existing\Class'
[2014-12-03 15:47:54] application.DEBUG: Attempting to reserve job from {queues} {"queues":"default"} []
PHP Fatal error:  Uncaught exception 'Resque\Exception\JobClassNotFoundException' with message 'Could not find job class Not\Existing\Class.' in /[...]/resque/vendor/vend/resque/src/Worker.php:175
Stack trace:
#0 /[...]/resque/vendor/vend/resque/src/Worker.php(448): Resque\Worker->createJobInstance('default', Array)
#1 /[...]/resque/vendor/vend/resque/src/Worker.php(278): Resque\Worker->reserve()
#2 /[...]/resque/bin/deamonize(19): Resque\Worker->work()
#3 {main}
  thrown in /[...]/resque/vendor/vend/resque/src/Worker.php on line 175

Fatal error: Uncaught exception 'Resque\Exception\JobClassNotFoundException' with message 'Could not find job class Not\Existing\Class.' in /[...]/resque/vendor/vend/resque/src/Worker.php on line 175

Resque\Exception\JobClassNotFoundException: Could not find job class Not\Existing\Class. in /[...]/resque/vendor/vend/resque/src/Worker.php on line 175

Call Stack:
    0.0002     232880   1. {main}() /[...]/resque/bin/deamonize:0
    0.0202    1831872   2. Resque\Worker->work() /[...]/resque/bin/deamonize:19
  950.4664    2212048   3. Resque\Worker->reserve() /[...]/resque/vendor/vend/resque/src/Worker.php:278
  950.4677    2212880   4. Resque\Worker->createJobInstance() /[...]/resque/vendor/vend/resque/src/Worker.php:448

... and boom workers fails miserably, process gets killed. bummer. Original fork did a great job with those cases it did break out just fine and marked job as failed - which I think is proper behavior. Now you get job lost and there is no trace of failure in resque gui for example. Obviously this can be taken care of by supervising tools with separate logging etc. But that does not feel right imho.

wotek commented 9 years ago

Got this fixed in my fork - just need to adjust tests etc. ;)

dominics commented 9 years ago

Tracked as #10 as well

dominics commented 9 years ago

Merged into master. Going to look at a test of the failure backend to prevent this recurring, and we need a test that mocks what the Resque GUI does with munging the payload/ID.