cjslack / grok-debugger

MIT License
135 stars 31 forks source link

Add support for structured JSON? #2

Open mdschmitt opened 3 years ago

mdschmitt commented 3 years ago

First off, this is a fantastic app!! The completion especially awesome.

So, I wanted to ask if it'd be possible to add the capacity for returning structured JSON similar to how Logstash/Elastic do their parsing. For example, the following grok pattern would create the JSON below for the associated log line (notice the dots in the grok pattern): Sample Data:

28.28.28.28, 64.64.64.64 10.10.23.6 - - [20/May/2021:05:36:31 +0000] "GET /api/v3/myEndpoint?timestamp=1621488990895&limit=1000&uuid=ab90e750c9484aa19cd46929b58d392a HTTP/1.1" "200" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Electron/8.5.5 Safari/537.36"

Grok Pattern:

%{IPORHOST:http.client}, %{IPORHOST:http.proxy} %{IPORHOST:remote_addr} - (-|%{DATA:user.name}) \[%{HTTPDATE:nginx.access.time}\] "%{URIPROTO:request.type} %{URIPATH:uri_path}\?%{GREEDYDATA:uri_query} HTTP/%{NUMBER:http.version}" "%{NUMBER:http.response.status_code:long}" "(-|%{DATA:nginx.user_agent})"
{
  "remote_addr": "10.10.23.6",
  "request": {
    "type": "GET"
  },
  "uri_path": "/api/v3/myEndpoint",
  "uri_query": "timestamp=1621488990895&limit=1000&uuid=ab90e750c9484aa19cd46929b58d392a",
  "nginx": {
    "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Electron/8.5.5 Safari/537.36",
    "access": {
      "time": "20/May/2021:05:36:31 +0000"
    }
  },
  "http": {
    "proxy": "64.64.64.64",
    "client": "28.28.28.28",
    "version": "1.1",
    "response": {
      "status_code": 200
    }
  }
}
gitmopp commented 1 year ago

Support for nested field will be helpful, e.g. not all patterns defined https://github.com/cjslack/grok-debugger/blob/master/public/patterns/httpd will work. The tool does not understand the syntax [source][address] in HTTPD_COMMONLOG %{IPORHOST:[source][address]} ....