search

MultiQC / MegaQC

Web application to collect and visualise data across multiple MultiQC runs.
http://megaqc.info/
GNU General Public License v3.0
94 stars 28 forks source link

Bug report - Overlapping scheduler jobs cause conflict #526

Open djwooten opened 2 months ago

djwooten commented 2 months ago

Describe the bug

I have several reports in the Uploads table that have status = "IN TREATMENT" and message = "The document has been uploaded successfully". Looking at upload_reports_job() this shouldn't be possible, as that message is only set if the status is simultaneously set to "TREATED".

To Reproduce

I believe that what has happened is

1) Too many reports were added to the uploads/ directory. 2) The scheduler job (job 1) starts processing these reports. 3) Before job 1 is done, a second scheduler job (job 2) begins that tries to process some of the same reports that are still listed as "NOT TREATED", because job 1 hasn't gotten to them yet. 4) One of the jobs processes a report, sets it to status="TREATED" and message="The document has been uploaded successfully", and then deletes the file before the other job gets to it. 5) When the other job gets to the same report, it updates the row to "IN TREATMENT" again, and then crashes when it tries to load the file to determine gzipped status, because the file doesn't exit anymore.

Expected behavior The reports should not show up as pending reports with "IN TREATMENT" status, but rather should be "TREATED".

System Details