TromsFylkestrafikk / ragnarok

Harvest public transport data for statistics usage
0 stars 0 forks source link

Caneled batches during max failures doesn't reset batch info #80

Closed tfk-kaare closed 2 months ago

tfk-kaare commented 3 months ago

When a running batch job reaches max failures and cancels itself, the batch info attached to chunks isn't properly reset. It seems only the 'failed' part of the operation (fetch or import) resets the accompanying batch column.

tfk-kaare commented 3 months ago

The problem is complex. The ChunkDispatcher::dispatchJobs() fires away a job, which is executed from background queue runners. Two of them. Both jobs are executed and both may fail, causing the Batch::except() trigger to run. But we limit the number of errors allowed in a batch run through the SkipOnBatchErrorLimit middleware. Jobs are cancelled here. Cancelling jobs will not fire the Bus::finally() handler set up in dispatchJobs(). Therefore we need to reset and communicate to the front end about reset jobs in a separate abstraction/layer.

So far, during testing of this I've reset the chunk batch info manually and fired the required broadcast event wherever suitable, but this needs to be ripped out to a wrapped service/class/handler of sort.