elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.76k stars 8.16k forks source link

[Grok Debugger] Support debugging multiple lines of sample log data #114502

Open smokris opened 2 years ago

smokris commented 2 years ago

When I'm constructing a grok pattern, I often want to test it on multiple similar lines of log data, to ensure my pattern matches all of them. Currently Kibana's Grok Debugger applies the Grok Pattern just once to the Sample Data. I'd like to be able to paste multiple lines of log data into the Sample Data field, and after I click the Simulate button, see an array of objects in the Structured Data section.

For example, I'd like to be able to paste these log lines into Sample Data:

2021-08-20T10:26:20-04:00 NOTICE: HORDE Guest user is not authorized for Filters (Host: 1.2.3.4). [pid 72328 on line 324 of "/usr/share/pear/Horde/Registry.php"]
2021-09-02T14:15:41-04:00 WARN: HORDE PHP ERROR: fgets(): SSL: Connection reset by peer [pid 98563 on line 156 of "/usr/share/pear/Horde/Imap/Client/Socket/Connection/Socket.php"]
2021-10-10T17:36:31-04:00 NOTICE: HORDE User foo is not authorized (Remote host: cpe-001-002-003-004.dhcp.example.com) [pid 12172 on line 324 of "/usr/share/pear/Horde/Registry.php"]

and this Grok Pattern:

%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:[log][level]}: HORDE %{DATA:message} \((Host|Remote host): %{IPORHOST:[client][ip]}\).? \[pid %{NUMBER:[process][pid]} on line %{NUMBER:[log][origin][file][line]} of "%{DATA:[log][origin][file][name]}"\]

and see a result like this:

[
  {
    "[client][ip]": "1.2.3.4",
    "[process][pid]": "72328",
    "[log][level]": "NOTICE",
    "[log][origin][file][name]": "/usr/share/pear/Horde/Registry.php",
    "[log][origin][file][line]": "324",
    "message": "Guest user is not authorized for Filters",
    "timestamp": "2021-08-20T10:26:20-04:00"
  },
  {
    "tags": ["_grokparsefailure"]
  },
  {
    "[client][ip]": "cpe-001-002-003-004.dhcp.example.com",
    "[process][pid]": "12172",
    "[log][level]": "NOTICE",
    "[log][origin][file][name]": "/usr/share/pear/Horde/Registry.php",
    "[log][origin][file][line]": "324",
    "message": "User foo is not authorized",
    "timestamp": "2021-10-10T17:36:31-04:00"
  }
]
elasticmachine commented 2 years ago

Pinging @elastic/kibana-stack-management (Team:Stack Management)

elasticmachine commented 2 weeks ago

Pinging @elastic/kibana-management (Team:Kibana Management)