Closed sanel closed 2 years ago
I adjusted the error messages slightly to debug this further since subreddit / comment parsing were using identical messages. Then I added the error message returned by json_tokener_error_desc
, which turned all of these errors into the following:
** (process:13196): WARNING **: 21:52:15.978: failed to parse comment json: nesting too deep
This is because json-c by default only allows JSON objects to be nested with a depth of around 32 (JSON_TOKENER_DEFAULT_DEPTH
).
Either json_object_from_fd_ex
or json_tokener_new_ex
needs to be used to override this depth limit to some higher (but still finite) value. Such a change will also have to be reflected when reading back from the spool file as well.
I'll try to write a patch this weekend.
Thanks for tracking it down! Let's try doubling it and see what happens.
When https://reddit.com/r/wallstreetbets is select as group, nntpit will report a lot of warnings, like:
but when you go to
/r/programming
, it will print a lot less warnings:I presume this has to do with number for posts in each thread? /r/wallstreetbets is overcrowded these days...
nntpit is started with:
./nntpit -D -t 3 -p 8119
.