search

statisticalbiotechnology / triqler

The triqler (TRansparent Identification-Quantification-linked Error Rates)'s source and example code
Apache License 2.0
19 stars 9 forks source link

Multiprocessing fails on Windows #8

Closed MatthewThe closed 5 years ago

MatthewThe commented 5 years ago

Running on a windows machine with Anaconda+Python3 results in the following error when trying to calculate protein posteriors:

AttributeError: Can't pickle local object 'MyPool.__init__.<locals>.<lambda>'

This is a well known problem with the multiprocessing module (e.g. here). The trick is apparently to define the function as a "top-level" function, which might be a bit cumbersome in the current state of the code.

MatthewThe commented 5 years ago

Funnily enough it was actually the lambda function defined here that caused the problem: https://github.com/statisticalbiotechnology/triqler/blob/08aff090bad1b950bec95a9d56b9a279bd6930ff/triqler/multiprocessing_pool.py#L10

Simply creating a named function for this lambda function seems to solve the problem.