search

allegro / node-worker-nodes

A node.js library to run cpu-intensive tasks in a separate processes and not block the event loop.
Apache License 2.0
489 stars 33 forks source link

bson buffer overflow #10

Closed noam-almog closed 7 years ago

noam-almog commented 7 years ago

Hi, Right now this library is using bson from mongo. there is a problem with the library here. Bson is using a predefined buffer with max size of 17mb, once this buffer is over node will shut down the process for security reasons without any way for the system to recover from it (no process events, try catch or anything else i could think of). There aren't many ways to resolve this, bson does not allow us to modify the buffer value which leaves us with two basic options:

  1. switch bson with something else, MessagePack can be a viable option (and a simple drop in replacement)
  2. leave things the way they are but allow to customize the message serializer (leaving the door open for many other binary serializers, like protobuf).
kwiatkk1 commented 7 years ago

Hi Noam,

thanks for your research and the code! :) BSON was originally picked because it is binary and schema-less. Replacing it with MessagePack sounds good to me, as it can also be used in a schema-less manner that allows passing arbitrary messages. So I'm for option 1.

I will check in the evening if everything works fine after the switch and I will merge your pull-request then. Thanks!