Open ronag opened 1 week ago
This can be pretty interesting, we can gain some bit in performance.
Tho the serialization/deserialization step might become the bottleneck isn't it? Or how do you propose to handle the scenario of sending JS objects?
Here it will mean to maintain two shared buffers to keep state and for data synchronization, do you see an issue of going out of bounds?
Part of the bottlenecks for Piscina at the moment is that we need to transfer data to the worker. Which is not efficient with the current implementation in NodeJS. This means that the tasks sent to piscina need to be large enough that the overhead of sending the task is not too bug.
One way around this would be to avoid the
postMessage
API's and use a ring buffer to write the tasks + data and let the user provide a Buffer serialize/deserialize function.Here is a ring buffer implementation we have been using internally for inspiration: