axiomhq / axiom-syslog-proxy

A syslog push interface to Axiom.
https://axiom.co
MIT License
9 stars 4 forks source link

Add more robust json encoding support #16

Closed gordallott closed 3 years ago

gordallott commented 3 years ago

this commit enables the usecase of json messages being encoded into the MSG field of a syslog message, for example:

<12> {"foo": "bar"}
<14>Jan  1 14:40:51 host app[24]: {"foo": "bar"}

closes axiomhq/axiom#2250


this ended up being a little bit more awkward than I would like, but functionally it became a case of

  1. look for an early indicator of json being a part of the line, } near the end is a good indicator
  2. look for a matching start brace, this doesn't mean we have valid json it just means this is probably json
  3. ignore syslog PRE values when looking for a start match for <12> {} situations
  4. if we found a start and end brace, try and parse as json and fallback to syslog on error
  5. no start and end brace then just parse syslog
  6. run the same detection on the MSG value of the parsed syslog, merge the json attributes with the syslog attributes

the extra detection steps are going to slow parsing but benchmarking suggests that this only becomes measurable in special cases like extreme levels of whitespace padding

codecov[bot] commented 3 years ago

Codecov Report

Merging #16 (9a681fe) into main (cc66c4c) will increase coverage by 0.68%. The diff coverage is 78.94%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #16      +/-   ##
==========================================
+ Coverage   73.57%   74.26%   +0.68%     
==========================================
  Files           6        6              
  Lines         704      746      +42     
==========================================
+ Hits          518      554      +36     
- Misses        136      140       +4     
- Partials       50       52       +2     
Impacted Files Coverage Δ
parser/app_parsers.go 12.50% <0.00%> (ø)
parser/parse.go 78.64% <79.06%> (+1.34%) :arrow_up:
parser/api.go 27.94% <84.61%> (+13.39%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update cc66c4c...9a681fe. Read the comment docs.