anilmaurya / fast_jsonparser

Fastest Json parser for Ruby, wrapper for simdjson
MIT License
307 stars 10 forks source link

Use a single static parser for both performance and avoiding leaks #12

Closed casperisfine closed 4 years ago

casperisfine commented 4 years ago

@XrXr pointed out to me that rb_raise isn't quite safe in C++. Long story short rb_raise bypass the C++ destructors which can lead to various leaks etc.

Notably simdjson::dom::parser has a destructor it uses to deallocate the memory it uses. I attached the leak.rb script, if you run it against maser you'll see memory usage go through the root very quickly.

The good new is that simdjson::dom::parser is designed to be re-used. When you call it again it re-use the memory it previously allocated, so by instantiating a single parser once, and then using only that one, we avoid leaks.

The added benefit is that it should be slightly more performant as parse, load, and load_many no longer need to build the parser.

Ref: https://github.com/simdjson/simdjson/blob/master/doc/performance.md#reusing-the-parser-for-maximum-efficiency Also note that this strategy can work because simdjson is thread safe for now (https://github.com/simdjson/simdjson/issues/681), if that change we might need a different strategy.

anilmaurya commented 4 years ago

Thank you for fixing this issue, I really appreciate your contribution 🏅