Open markfinger opened 8 years ago
Didn't take much poking through the issues for each of the threading libraries before issues start emerging (lack of event loop and primitives - setTimeout, etc).
That being said, I did come across one exciting little thing: https://github.com/nodejs/node/pull/2133#issuecomment-222754118
Something I've been struggling to handle cleanly is how to handle multiple phases with workers.
The use-case that's rolling around involves an initial transformation/dep-analysis phase, followed by contextual rewrites of the file (dep paths, inlining assets, etc).
Fundamentally, it looks like a question of how to handle resumable/iterative jobs across separate processes?
Probably need some notion of jobs where we can handle multiple iterations of work on assets. A key part of this would be preserving references to expensive data (ASTs, for example) and trying to minimise transport overhead. Also need to cover invalidating records.
This is an interesting read on threading in Node http://neilk.net/blog/2013/04/30/why-you-should-use-nodejs-for-CPU-bound-tasks/
In general, it seems more practical these days:
Though, there is still the usual caveat with JS workers that they only evaluate scripts, so you need to build libraries with webpack/browserify.
The big benefit of threads (vs node's cluster or fork) would be reduced memory usage. A common problem that we've hit on servers (both during CI and deployment) is that we blow out the VM's memory limit and the build process dies halfway. Threads might be a solution to this problem.