search

celery / billiard

Multiprocessing Pool Extensions
Other
420 stars 251 forks source link

Add on_ready_counter to Worker.__reduce__. #400

Closed Androidown closed 10 months ago

Androidown commented 10 months ago

We need to add on_ready_counter to Worker.__reduce__, otherwise a spawned worker will have on_ready_counter == None

Androidown commented 10 months ago

looks OK but would be better if you can share bit more context here?

I'm trying to use SpawnWorker in celery, it works fine unless occasionally a WorkerLostError exit code 155 is raised. After some investigation, I found the error was already fixed in this PR. But because on_ready_counter is not present in Worker.__reduce__, it could not be correctly seriliazed and deserialized into a subprocess using spawn (It works for a forked subprocess though). The test added will raise AttributeError: 'NoneType' object has no attribute 'value' if on_ready_counter is not included.