search

logstash-plugins / logstash-filter-dissect

Extract structured fields from an unstructured line
Apache License 2.0
13 stars 22 forks source link

remove_field is executed before dissection (regression from version 7.8.1 -> 7.10.1) #75

Open fkellner opened 3 years ago

fkellner commented 3 years ago

The following configuration worked fine in Logstash Version 7.8.1:

  dissect {
    mapping => {
      "message" => "%{}<Stuff>%{tables}<other Stuff>%{}"
    }
    remove_field => ["message"]
  }

After upgrading to 7.10.1, it produced a Dissectfailure/Warning that the "message"-field could not be found.

I tested with the file output and the message field was part of the event before it reached the dissect block. After moving the remove_field option to a later block, everything works fine again, so I would guess that the execution order of the common options has changed to before the filter.

I do not know if I should file it here or it is a logstash issue - personally, I think being able to remove a field after dissecting it without needing to open a new filter block was the more readable solution.