KiteSDK Grok issue

[imoreno-stratio]

When using "OR" expressions in a grok template, and using the same field to be extracted in these different OR-ed expressions, field will be set only when matching the first expression, otherwise field will not be available. Example: grok { dictionaryString: """ DATA . CSVF [^,] CSVFQ [^'] CSVFDQ [^"]""" expressions: { message: """("%{CSVFDQ:firstName}"|'%{CSVFQ:firstName}'|%{CSVF:firstName})%{DATA:extra}""" } }

• Values and results using Kite SDK morphline core v 0.17.0 (failing, firstName only set in third case, matching first OR-ed expresion): Event: { headers:{message=Name no quotes,extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1230.csv, extra=,extradata} body: } Event: { headers:{message='Name simple quotes',extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1230.csv, extra=,extradata} body: } Event: { headers:{message="Name double quotes",extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1230.csv, extra=,extradata, firstName=Name double quotes} body: }

If a different field name is used in these OR-ed expressions, it works fine and OR-ed expressions are evaluated: message: """("%{CSVFDQ:firstName}"|'%{CSVFQ:firstName2}'|%{CSVF:firstName3})%{DATA:extra}"""

Event: { headers:{message=Name no quotes,extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1231.csv, extra=,extradata, firstName3=Name no quotes} body: } Event: { headers:{message='Name simple quotes',extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1231.csv, extra=,extradata, firstName2=Name simple quotes} body: }
Event: { headers:{message="Name double quotes",extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1231.csv, extra=,extradata, firstName=Name double quotes} body: }

If Kite SDK morphline core v1.1.0 is used instead v0.17.0, it works as expected: Event: { headers:{message=Name no quotes,extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1232.csv, extra=,extradata, firstName=Name no quotes} body: } Event: { headers:{message='Name simple quotes',extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1232.csv, extra=,extradata, firstName=Name simple quotes} body: } Event: { headers:{message="Name double quotes",extradata, file=/opt/sds/ingestion/examples/hola-tests/./spooldir/CM_TestBug_1232.csv, extra=,extradata, firstName=Name double quotes} body: }

We suggest to upgrade Kite SDK dependencies to 1.1.0 in order to solve this issue.

Thanks.

[aargomaniz]

Thanks for your feedback Ivan, we will update the dependency in the next release.

[epeinado]

Hi Ivan! We have just updated Kite SDK dependency (you have it in Master branch). Thanks for your help :)