josdejong / workerpool

Offload tasks to a pool of workers on node.js and in the browser
Apache License 2.0
2.04k stars 148 forks source link

Worker thread crashed silently or killed by OS while pool.stats considers it as active task #431

Closed itsdheerajbudhiraja closed 5 months ago

itsdheerajbudhiraja commented 5 months ago

I am using workerpool to validate a large amount of data (around 20k records array). After validating each record, there is a database call for each record also inside the worker code

After processing a few records, Logs from worker thread stops printing and also CPU usage goes down indicating worker thread has crashed. I also have setInterval method in the main thread which prints Pool stats and it is printing active thread as 1.

In logs I don't see any std err from the worker thread.

How can I determine the root cause for this?

Is there any process.on("error" | "sigterm" | "sigkill") method I can pass to the worker thread to handle OS signals if the OS is terminating that thread abruptly?

josdejong commented 5 months ago

Some ideas for debugging: