Open ChrisSmith opened 4 years ago
isValid
and parse
don't seem affected, so its specific to lazyParse
Hi @ChrisSmith, thanks for your input. I will investigate that and get back to you, cheers!
Hi @ChrisSmith, I took some time to investigate that today and in my machine (2018 Macbook i7 with 16GB RAM) it doesn't leak memory as we would expect, but had a peak of around 5GB but soon released it and had an average usage of 2GB. This on 7 threads as I didn't find a way to control the number of threads from within benchmark
lib.
We also have just updated the library. Now it's currently on version 0.5.0
, so it could've been a bug on the upstream.
Please also note that simdjson.lazyParse
and simdjson.isValid
should do basically the same work, although we keep a pointer to the parsed object here. But that is always released here once the record (pointer owner) is removed from the stack.
I will need more info from you, if it still happens. Thanks!
I can also confirm the memory leak.
When running the benchmarks locally, the process gradually consumes all memory on my machine until it runs out of memory (~28GB)
Commenting out
simdjson.lazyParse
in the benchmark removes leak behavior, so I don't think its the test setup. Printing the node heap statistics shows that it is stable ~80 mb, so the leak is likely in the c++ code.