search

celery / billiard

Multiprocessing Pool Extensions
Other
419 stars 252 forks source link

Fix for crash on windows #225

Closed SPKorhonen closed 7 years ago

SPKorhonen commented 7 years ago

As there is no fork support on windows pickle is used to transfer the process state. Unfortunately pickle can't handle nested classes causing a crash. FIX: Moved one boolean variable to process instance variable and removed nested class from pool

SPKorhonen commented 7 years ago

Billiard Pool uses nested class Process to add boolean attribute controlledtermination to process class. When starting worker processes, if no fork support is available (which is the case under Windows), the process state is pickled which in turn fails with "AttributeError: Can't pickle local object 'Pool.init__..Process'" due to nested class Process pickling of which is not supported. By moving the boolean flag to BaseProcess object instance variable this problem can be resolved.

Corresponding issue is #226. Also, for additional discussion about this issue, see https://github.com/celery/celery/issues/3821

SPKorhonen commented 7 years ago

For test, should just add a minimal test which tries to create Pool object under windows? Is this sufficient and/or sensible?

auvipy commented 7 years ago

yes provide a test that help us reproduce the issue :)

SPKorhonen commented 7 years ago

I've added rather rudimentary test but it get the job done. It reproduces the problem and allows for checking of the fix