Closed wojtekKrol closed 7 months ago
I guess you can call .terminate()
on the workerpool
to kill all workers, and then create a new workerpool
.
@josdejong I would like to make that logic, that worker process logic inside it (or best repeat it N times), and after that it will be terminated and re-created (with reset N counter) automatically.
I think what you can do is create a little wrapper function around your workerpool that:
N
tasks per worker)There is no support for terminating a single worker, but this would terminate all of them and re-create them once in a while to solve the memory leaks issue.
Description
I am using the
workerpool
library to manage multiple services in a Node.js application, specifically for crawling tasks using Playwright. However, I've encountered an issue with Playwright related to memory leaks. This seems to be a common problem among developers using Playwright, and the suggested workaround involves restarting the Playwright process to free up memory.Issue
In my application, each service is a separate worker within
workerpool
. One of these services, a crawler, is responsible for handling thousands of URLs. Due to the memory leak in Playwright, I need a way to programmatically restart this specific service (crawler) withinworkerpool
. The service is stateless and does not process any data persistently, so it should be feasible to restart it without losing important information.Current Implementation
Here is a simplified version of how the services are structured:
Request
I am seeking guidance or a feature within
workerpool
that would allow me to restart a specific service (especially the crawler service using Playwright) to handle the memory leak issue. This would involve terminating and then reinitializing the service's process. Any suggestions or solutions for this scenario would be greatly appreciated.