For some kinds of invalid JSON documents we are able to get a valid iterator to the illegal JSON document which later throws when we access a malformed field in the JSON document. Since we don't check if fields are valid in the parse_line function as we access them, and we don't have a try/catch around parse_line the exceptions goes uncaught and we lose out on properly reporting where the error occurred in the source file.
CLP version
0.1.2
Environment
ubuntu focal dev container
Reproduction steps
Ingest illegal JSON documents like
{"invalid": 17
and observe uncaught exception without log message telling us what file being ingested caused the issue.
Bug
For some kinds of invalid JSON documents we are able to get a valid iterator to the illegal JSON document which later throws when we access a malformed field in the JSON document. Since we don't check if fields are valid in the parse_line function as we access them, and we don't have a try/catch around parse_line the exceptions goes uncaught and we lose out on properly reporting where the error occurred in the source file.
CLP version
0.1.2
Environment
ubuntu focal dev container
Reproduction steps
Ingest illegal JSON documents like {"invalid": 17 and observe uncaught exception without log message telling us what file being ingested caused the issue.