We use a pseudo-parser to identify the json(s) inside the execution logs, then feed only those substrings to yojson for further analysis. However our custom parser is a bit too simple to work for everybody: It only checks that the "json" has correctly balanced parentheses {}[]"" (= it's a bit too generous). As a result, we silently ignore the wrongly identified json when yojson crashes and are unable to confidently report the mistakes to the user (because it's most likely our fault).
The ocaml compiler and ocamlformat can output lines that starts with a { but what follows is not json (for example https://github.com/ocurrent/current-bench/issues/321 )... A { should be followed by " or }, and our parser shouldn't recognize {{{ this is not json }}} and variations on this theme
Since our pseudo-parser is so fragile, we restricted it to identify json only when it starts at the very beginning of the line (so right after a \n). It would be nice if we could be more flexible here! (because letting the user prefix its json output can enable him to grep '^current-bench' and so forth)
ElectreAAS
commented
2 years ago
We use a pseudo-parser to identify the json(s) inside the execution logs, then feed only those substrings to yojson for further analysis. However our custom parser is a bit too simple to work for everybody: It only checks that the "json" has correctly balanced parentheses
{}[]""(= it's a bit too generous). As a result, we silently ignore the wrongly identified json when yojson crashes and are unable to confidently report the mistakes to the user (because it's most likely our fault).{but what follows is not json (for example https://github.com/ocurrent/current-bench/issues/321 )... A{should be followed by"or}, and our parser shouldn't recognize{{{ this is not json }}}and variations on this theme\n). It would be nice if we could be more flexible here! (because letting the user prefix its json output can enable him togrep '^current-bench'and so forth)