thekrakken / java-grok

Simple API that allows you to easily parse logs and other files
http://grok.nflabs.com/
Other
358 stars 152 forks source link

Multiple duplicate named matches doesn't work like logstash #40

Closed sherzberg closed 7 years ago

sherzberg commented 8 years ago

In logstash, if you have multiple of the same named patterns, logstash will return an array of matched values. Is this something this library should support?

logstash

If I have a logstash config of:

input {
    stdin{}
}
filter {
    grok {
        match => { "message" => "%{INT:id} %{INT:id}" }
    }
}
output {
    stdout { codec => json }
}
123 456
{"message":"123 456 678","@version":"1","@timestamp":"2015-11-17T17:05:10.292Z","host":"spencerherzberg-mbp","id":["123","456"]}

java-grok

patterns.txt

INT (?:[+-]?(?:[0-9]+))

grokissue.groovy

@Grab(group='io.thekraken', module='grok', version='0.1.1')
import oi.thekraken.grok.api.Grok
import oi.thekraken.grok.api.Match

Grok g = Grok.create("patterns.txt")
g.compile("%{INT:id} %{INT:id}");

String log = "123 456";
Match gm = g.match(log);
gm.captures();

System.out.println(gm.toJson());

output

{"id":123}
sherzberg commented 8 years ago

For another datapoint, grokdebug does the same thing as logstash (I'm not sure if it uses the same grok library under the hood though).

screenshot 2015-11-18 10 32 29
scottxusayhi commented 8 years ago

@sherzberg I just checked out the head from master and the feature (put values of the same named patterns into an array) works.

sherzberg commented 7 years ago

Yea I retested with the latest from master and it now works like logstash does. Can version 1.1.5 be released?

sherzberg commented 7 years ago

Here is at least one commit for the fix: https://github.com/thekrakken/java-grok/commit/622bd9af2bfe4086ea4a9a704454f65f0f50adb6

trixpan commented 7 years ago

@sherzberg glad this was fixed. Mind closing the ticket?

@anthonycorbacho any chances of releasing a new version? Keen to use the library as part of Apache NiFi processor but trying to avoid additional unnecessary transitive dependencies (i.e. code before bd098c3a0d76bf9ba2b5b239e6cba34edb8fb82b).

Cheers

sherzberg commented 7 years ago

Closing because this is fixed here https://github.com/thekrakken/java-grok/commit/622bd9af2bfe4086ea4a9a704454f65f0f50adb6! Thanks