However, the finally block never gets called. It appears that task.pipeline_run() and/or other code in task.py are not handling exceptions correctly, and therefore the ThreadPool workers are not getting gracefully shut down.
I've attached the errors I get when interrupting a run with Ctrl-C: ruffus-stderr.txt
I modified
drmaa_wrapper.run_job_using_drmaa()to terminate running jobs when the pipeline gets interrupted:https://github.com/kemaleren/ruffus/commit/c41181a445be57dd77bb949495861bf9c075c63f
However, the
finallyblock never gets called. It appears thattask.pipeline_run()and/or other code intask.pyare not handling exceptions correctly, and therefore the ThreadPool workers are not getting gracefully shut down.I've attached the errors I get when interrupting a run with Ctrl-C: ruffus-stderr.txt