imTigger / laravel-job-status

Add ability to track Job progress, status and result dispatched to Queue.
MIT License
407 stars 60 forks source link

Updating Job status within transactions #9

Closed SlyDave closed 4 years ago

SlyDave commented 6 years ago

Having issues using this with transactions

$count = 100;
$this->setProgressMax($count);

DB::beingTransaction();
try {
    for ($i=1; $i <= $count; $i++) {
        $this->setProgressNow($i) // no easy way to do this outside the transaction! :(
        dump($i);
    }
}
catch (\Exception $e) {
    DB::rollBack();
    $this->setProgressNow(0);
    throw $e;
}
DB::commit();

Obviously, with this set up it goes from 0 to 100, because all progress counter is ticking up inside the transaction which doesn't commit until the end of the loop.

What I'm trying to achieve is a way to tick up the progress, whilst the actual work isn't committed until the end, and if an error occurs, set the progress back.

imTigger commented 6 years ago

DB::beingTransaction() seems to be connection-wide. There seems no way to commit immediately inside a transaction. Unless we open another connection, but still need some testing

SlyDave commented 6 years ago

Yeah, this is pretty much going to need a secondary connection to work. Then it's going to need some finally keyword play to handle errors in the executing job to set the status.

I wonder if it's possible to clone the connection based upon the existing connection configuration, otherwise we enter the territory of having to get people to configure a second connection just to use this - which isn't ideal...

tpetry commented 5 years ago

Hey, that's my problem, too: I have a job that performs operations on the database for up to 5 minutes and i would like to provide the user with progress information.

How about alternatively storing job status information in redis instead of the database? This would also solve the efficiency problem that when saving fine-granular Progress updates a lot of database updates take place.

A disadvantage of course is that no eloquent models can be used anymore. But since the model could not be changed anyway (#17) it is probably not a problem at all. There would only have to be a (static) function to get the current status information based on a job ID.

crashkonijn commented 5 years ago

We could do something like this, although I can't really oversee possible side effects of this:

$c1 = app('db.connection');
$c2 = clone $c1;

$c1->transaction(function () use ($c2) {
    $c2->table('job_statuses')->delete(); // this still deletes everything
});
imTigger commented 4 years ago

Transaction support is added in 1.1.0-rc1