timo / json_fast

a naive imperative json parser in perl6, to evaluate performance against JSON::Tiny
Artistic License 2.0
27 stars 20 forks source link

Memory leak #21

Closed AlexDaniel closed 7 years ago

AlexDaniel commented 7 years ago

Compared to JSON::Tiny it is leaking like hell. I noticed it after Pastebin::Gist switched back to JSON::Fast in https://github.com/zoffixznet/perl6-Pastebin-Gist/commit/45c25120b6b2e2c9e07f27440a8febbc05a2ab96.

I'm observing the issue through unicodable, but I guess you can also reproduce it separately.

unicodable6test: { 0 < $_ < 8000 }
<unicodable6test> naxieAlDle, U+0001 <control-0001> [Cc] (control character)
<unicodable6test> naxieAlDle, U+0002 <control-0002> [Cc] (control character)
<unicodable6test> naxieAlDle, 7999 characters in total: https://gist.github.com/a14b9d38cbdc7500c80cf32752efe0c9

… Meanwhile it is using several gigs of memory just because of this gist alone!
zoffixznet commented 7 years ago

Here's a snippet that repoes the Issue (which may or may not be a memory leak; just large memory use).

With JSON::Fast this snippet uses 2.3GB of RAM. But if you swap the decoder to JSON::Tiny, the use is only 173MB

use JSON::Fast;
use HTTP::UserAgent;

my $res = HTTP::UserAgent.new.get: 'https://gist.githubusercontent.com/zoffixznet/eada23f2d7488adfb1a735f65678bb3b/raw/8496c7daadd708a08944fd665d9cf65ab6de6b6c/gistfile1.txt';
fail $res.status-line unless $res.is-success;
$ = from-json $res.content;

say "Done working; measure the RAM!";
sleep;
timo commented 7 years ago

the same file from zoffix' gist now (as of version 0.9) completes in half a second no my machine and takes only 92 megs of ram at its peak