tstack / lnav

Log file navigator
http://lnav.org
BSD 2-Clause "Simplified" License
8.06k stars 324 forks source link

Pretty print regression for json logs #1259

Open malnoxon opened 7 months ago

malnoxon commented 7 months ago

lnav version Expected behavior verified on v10.0.1, issue encontered on v0.11.1 and v0.11.2

Describe the bug I have a log file that I have a custom formatter to parse. Previously it would successfully parse the file and I could then use the pretty print mode to pretty print the json blob in the context but after upgrading the parser parses it but pretty printing does nothing.

What I see in lnav on the raw log view 0.12.2 image

What I see on pretty print in ~0.11.2~ 0.12.2 image

What I see on 0.10.1 (and expect to see on later versions but don't) image

To Reproduce Create a new file ~/.lnav/formats/generic_json/format.json with the following contents

{
  "json_log": {
    "title": "JSON log",
    "description": "Generic JSON log",
    "json": true,
    "file-pattern": ".*\\.json$",
    "level-field": "level",
    "level" : {
      "critical" : "critical|emergency|alert",
      "error" : "error",
      "warning" : "warning",
      "info": "info",
      "debug": "debug",
      "notice": "notice"
    },

    "timestamp-field": "timestamp",
    "value": {
      "msg": {
        "kind": "string"
      }
    },
    "line-format": [
      {
        "field": "timestamp"
      },
      " ",
      {
        "field": "level",
        "min-width": 8
      },
      {
        "field": "msg"
      }
    ]
  }
}

lnav -i ~/.lnav/formats/generic_json/format.json

Create a log file:

{"timestamp":"2024-04-25T15:31:25.358+00:00","level":"warning","msg":"CSP Enforcement Violation","context":{"scope":null,"customerId":null,"organizationId":null,"userId":null,"blockedUri":"https:\/\/content.p.exmaple.com\/guide-content\/mpnuQuihIGVljTw7UavJObWIGXI\/d-1eKOtlbL7UuTdtNhCwEoNkuWQ\/L8Lno0giqdTqrkSHTSWpws7AbLU.dom.json?sha256=Sp2HCFH1JJoVtqk1AIdKGpCWjuhiYc7Ts_Py898hzeo","documentUri":"https:\/\/foo.example.com\/Dashboard\/index","violatedDirective":"connect-src"}}
{"timestamp":"2024-04-25T15:31:25.381+00:00","level":"info","msg":"ISI admin activity","context":{"scope":null,"customerId":null,"organizationId":null,"userId":null,"sessionId":"1buitqgqqbojmka72hu61fnadq","identityUserId":7,"contextCustomerId":1,"contextOrganizationId":null,"impersonator":null,"impersonatee":null,"method":"POST","path":"\/csp\/enforce","query":""}}

Open the log file with lnav <Shift>+P to try to pretty print

tstack commented 7 months ago

Just to clarify, I think you mean "0.12.2" and not "0.11.2", right? The first set of screenshots have changes that are present in the 0.12 series.

malnoxon commented 7 months ago

You're right, those screenshots were from 0.12.2.

Actual 0.11.2 screenshots: image

image

tstack commented 7 months ago

The pretty-print is not happening because the format does not set a body-field. In your format file, can you add "body-field": "msg" and try again.

I would say this particular case should work without the body field, though. Leave the bug open and I'll try to improve the situation.

malnoxon commented 7 months ago

Pretty printing does work with that change, thank you!