Adding a tool to help parsing Decision Engine structured logs

[mambelli]

Doing troubleshooting sometimes is useful to display just the event messages or other components. This can be done by ingesting the log files in Elastic Search and using some graphical/web tool but is nice also to have a quick local option.

[jcpunk]

Did you want to add this tool to setup.py's console_scripts?

[codecov[bot]]

Codecov Report

Merging #544 (fdf3556) into master (1da0d61) will increase coverage by 0.50%. The diff coverage is 99.08%.

@@            Coverage Diff             @@
##           master     #544      +/-   ##
==========================================
+ Coverage   84.22%   84.72%   +0.50%     
==========================================
  Files          48       49       +1     
  Lines        3093     3202     +109     
  Branches      481      508      +27     
==========================================
+ Hits         2605     2713     +108     
  Misses        454      454              
- Partials       34       35       +1     

Flag	Coverage Δ
python-3.10	84.60% <99.08%> (+0.47%)	:arrow_up:
python-3.6	84.36% <99.08%> (+0.45%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/decisionengine/framework/util/logparser.py	99.08% <99.08%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 1da0d61...fdf3556. Read the comment docs.

[retzkek]

Why not use an existing tool for this, e.g. https://github.com/egnyte/ax?

[mambelli]

Why not use an existing tool for this, e.g. https://github.com/egnyte/ax?

The current log format is hybrid, with some fields and the JSON dictionary. ax is not parsing it correctly. At least I was unable to make it work well. Some sample lines are in the fixture file committed.

[pep8speaks]

Hello @mambelli! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2021-11-03 14:40:11 UTC

[mambelli]

I have a problem w/ lines 177 and 179: black added a space before ":" in outline[len(argsparsed.separator) :] and flake8 does not like it. Python works either way. Personally, I'd prefer it w/o the space. Suggestions?

• Solved adding as suggested the directive to flake8 to ignore this.

[shreyb]

@mambelli I'm also getting a segfault in my unit tests now for #523 (no timeout anymore). I didn't early last week, so I'm curious as to what could be causing that (and have been trying all morning, unsuccessfully, to run it down).