scipion-em / scipion-pyworkflow

Underlying pyworkflow module for the Scipion framework
GNU General Public License v3.0
6 stars 5 forks source link

Step Queue executor: fix recursive use of cursors #535

Closed danielmarchan3 closed 2 months ago

danielmarchan3 commented 3 months ago

When sending the steps to the queue with thread parallelization, an error might occur due to the recursive use of cursors when updating the job IDs, as two steps could try to modify the database simultaneously. To handle this situation, we used a lock mechanism to ensure that only one thread can modify the database at a time, changing the Lock object to Rlock. Recursive locks allows a thread to acquire lock on same object more than one time, thus avoiding deadlock situation. This fixed the concurrency problems we had before.

sonarcloud[bot] commented 2 months ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

pconesa commented 2 months ago

Thanks Dani! Hopefully this will fix those unstability cases in streaming scenarios once for all.