search

pinojs / sonic-boom

Extremely fast utf8 only stream implementation
MIT License
261 stars 41 forks source link

Add a supported way to limit buffer size #117

Closed mmarchini closed 2 years ago

mmarchini commented 2 years ago

We've been facing some memory issues which we tracked down to be really huge sonic-boom buffers. To prevent this kind of issue, it would be nice if sonic-boom provided a way to limit the buffer size when running on async mode. Are there any preferences on how such a feature would be implemented? I could see it being implemented as a maxLength option that drops anything new if the buffer length goes past it, or as an optional handler function that allows developers to implement custom dropping logic. Personally I think I prefer maxLength.

I'm happy to implement it if this is something others would be interested on.

mcollina commented 2 years ago

Go for the second option, I prefer it as well! I I would adding this logic in https://github.com/pinojs/sonic-boom/blob/2edc3ae41fdc5269ed2038b4acb2cc71482e4cff/index.js#L225.

mmarchini commented 2 years ago

Ok, I'll start implementing it as the second option (handler function for custom dropping logic).

mcollina commented 2 years ago

I'm sorry, go for maxLenght!

mmarchini commented 2 years ago

Fixed in #119