Grok TransformItems task gives out blank lines when the log pattern does not match

[shrutimantri]

Expected Behavior

When the log lines do not match, the line should be completely ignored.

For the following input:

"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"
"2020-12-14T12:18:51.397Z INFO Your output is: 12345"
"2020-12-14T12:18:51.397Z WARN Warning message!!"
"2020-12-14T12:18:51.397Z ERROR Somethiong went wrong.."
"DEBUG:root:This message should go to the log file"
"ERROR:root:Got exception on main handler"Traceback
"Traceback (most recent call last):"
"  File \"/tmp/teste.py\", line 9, in <module>"
"    run_my_stuff()"
"NameError: name 'run_my_stuff' is not defined"
"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"

The expected output should be:

"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"
"2020-12-14T12:18:51.397Z INFO Your output is: 12345"
"2020-12-14T12:18:51.397Z WARN Warning message!!"
"2020-12-14T12:18:51.397Z ERROR Somethiong went wrong.."
"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"

Actual Behaviour

When the log lines do not match, we get a blank line for the non-matching line.

For the following input:

"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"
"2020-12-14T12:18:51.397Z INFO Your output is: 12345"
"2020-12-14T12:18:51.397Z WARN Warning message!!"
"2020-12-14T12:18:51.397Z ERROR Somethiong went wrong.."
"DEBUG:root:This message should go to the log file"
"ERROR:root:Got exception on main handler"Traceback
"Traceback (most recent call last):"
"  File \"/tmp/teste.py\", line 9, in <module>"
"    run_my_stuff()"
"NameError: name 'run_my_stuff' is not defined"
"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"

We get the following output:

"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"
"2020-12-14T12:18:51.397Z INFO Your output is: 12345"
"2020-12-14T12:18:51.397Z WARN Warning message!!"
"2020-12-14T12:18:51.397Z ERROR Somethiong went wrong.."
<blank line>
<blank line>
<blank line>
<blank line>
<blank line>
<blank line>
"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"

Steps To Reproduce

1. Run the flow in the example section.
2. Use the file with the following contents as input:

"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"
"2020-12-14T12:18:51.397Z INFO Your output is: 12345"
"2020-12-14T12:18:51.397Z WARN Warning message!!"
"2020-12-14T12:18:51.397Z ERROR Somethiong went wrong.."
"DEBUG:root:This message should go to the log file"
"ERROR:root:Got exception on main handler"Traceback
"Traceback (most recent call last):"
"  File \"/tmp/teste.py\", line 9, in <module>"
"    run_my_stuff()"
"NameError: name 'run_my_stuff' is not defined"
"2020-12-14T12:18:51.397Z INFO Hi from Kestra team!"

Environment Information

• Kestra Version: 0.18.1
• Plugin version: 0.18.1
• Operating System (OS / Docker / Kubernetes): Docker
• Java Version (If not docker): N/A

Example flow

id: grok-flow
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: grok
    type: io.kestra.plugin.transform.grok.TransformItems
    pattern: "%{TIMESTAMP_ISO8601:logdate} %{LOGLEVEL:loglevel} %{GREEDYDATA:message}"
    from: "{{ inputs.file }}"