IBM / db2sock-ibmi

An asynchronous PASE Db2 and IBM i integration library
MIT License
4 stars 7 forks source link

Crash when parsing a big json file #25

Closed kadler closed 6 years ago

kadler commented 6 years ago

Original report by Teemu Halmela (Bitbucket: teemu_, GitHub: Unknown).


If I try to parse this json file the toolkit crashes. This is the same test as j0185 except it has newlines and spaces removed.

It crashes because bigkey->count isn't incremented when bigkey is expanded and this mismatch compounds over time and things start to crash. The normal version isn't crashing because the count is incremented on empty chars, so the bigkey will grow (unnecessarily) and we have enough room to put all the real values.

By discarding the empty chars (code) the normal version will also crash.

To fix this crash we can apply this fix.

Other way and maybe better, so we don't need the empty char stuff, might be this fix.

kadler commented 6 years ago

Original comment by Teemu Halmela (Bitbucket: teemu_, GitHub: Unknown).


Thank you, this implementation seems fine. No more crashes.

kadler commented 6 years ago

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).


Ok, fix for json_grow_key. Thanks.

#!bash

bash-4.3$ ./test1000_sql400json32 ../json/j0186_pgm_hamela04-ds-rpg-occurs-500-slim  
input(5000000):
{"pgm":[{"name":"HAMELA04","lib":"DB2JSON"},

:

arm11":"äöÄÖåÅáÁàÀ"}]}]}

result:
success (0)

Note: I used your idea for count (k). However, I wanted to keep the key count and max internals 'encapsulated' in json_grow_key (C++ish). Thanks.