eastgenomics / eggd_artemis

Creates download urls for files required and outpus in a text file to be shared with the scientists
0 stars 0 forks source link

Bug in handling exceptions in concurrent.futures #26

Open jethror1 opened 3 weeks ago

jethror1 commented 3 weeks ago

If an exception is raised by the function called in concurrent.futures here it will incorrectly raise a TypeError:

https://github.com/eastgenomics/eggd_artemis/blob/d31e869494fa3f441ba17657f34c23cb5b40974d/src/eggd_artemis.py#L158-L160

This is from when the concurrent jobs are submitted to the ThreadPool the submission creates a set and not a dict:

https://github.com/eastgenomics/eggd_artemis/blob/d31e869494fa3f441ba17657f34c23cb5b40974d/src/eggd_artemis.py#L135-L137

This instead should store the future as the key and the param input as the value:

concurrent_jobs = { 
     executor.submit(_find, report): report for report in reports 
 } 

Relevant example in docs: https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor-example