search

chjj / bthreads

worker threads for javascript
Other
48 stars 0 forks source link

Add an "inline/run-in-main-process" backend? #2

Open Apollon77 opened 5 years ago

Apollon77 commented 5 years ago

Hi,

what about also adding an "inline/run-in-main-process" backend which just executes the code inside the current process and event loop? SO kind of also executing the "worker process" locally in the same instance. Because with this it would be completely flexible then.

My usecase is that we have Objects and processes that are running fine in the normal event loop normally, but when using worker_threads it could be a benefit because they have their own event-loop, but do not need the full memory of an own nodejs process. This is better for high-usage scenarios. Having them run as "cheild_process" spawn/fork also works but adds a full process memory footprint (which may conflict with low-mem systems like raspis or such).

Do also having the option to use the same interface to have the code executed "in the local event loop" could be really beneficial for us.

What do you think of that?

naz commented 3 years ago

Hey @Apollon77 , I think running a task in the parent process' even loop should be solved on a different layer, for example a job queue module which uses bthreads. This library seems to focus on polyfilling threads. Running code in the same process/even loop is on the client library imo.

To add an example to this, if you are using bree as job queuing library I think it would benefit to add some sort of "run inline" flag to job objects so that they would be executed "inline" instead of spawning a child process/worker thread.

@Apollon77, what approach are you using for job queue management?

Apollon77 commented 3 years ago

Maybe you are right ... We do not have "job queue" management really ... we allow to start "kind of" plugins in our project. They are normally forked as new processes. For systems with not that much RAM (ok in the past, maybe not that relevant anymore for all cases) we were searching for a way to "just" run them sandboxed with at least additional resource needs as possible.