Mutate ignores the configuration's operator order

[jordansissel]

(This issue was originally filed by @ndelucin at https://github.com/elastic/logstash/issues/1877)

---

Hi,

I discovered a strange issue regarding mutate evaluation. Don-t know if its a bug or a normal behavior of the Logstash "interpreter"...

Let me explain... I have a "hostname" field containing a FQDN such like "mycomputer.my.domain.com" Aim is to "clean" this field to only store the computer name (1st part of FQDN): hostname="mycomputer.my.domain.com" => filtering => hostname="mycomputer"

So I applied the following filter{} in my logstash conf file:

filter {
    mutate {
        split => ["hostname", "."]
        add_field => { "shortHostname" => "%{hostname[0]}" }
        rename => ["shortHostname", "hostname" ]
    }

but, it did not work at all. Here is stdout:

{
"message" => "a\r",
"@version" => "1",
"@timestamp" => "2014-10-10T15:37:57.485Z",
"hostname" => [
        [0] "mycomputer",
        [1] "my",
        [2] "domain",
        [3] "com"
],
"host" => "LFR003256",
"shortHostname" => "mycomputer"
}

_BUT !!_ If I re-arrange instructions into 2 distinct mutate blocks like this:

filter {
    mutate {
        split => ["hostname", "."]
        add_field => { "shortHostname" => "%{hostname[0]}" }
    }

    mutate {    
        rename => ["shortHostname", "hostname" ]
    }
}

Then it works like a charm:

{
"message" => "a\r",
"@version" => "1",
"@timestamp" => "2014-10-10T15:32:57.077Z",
"hostname" => "mycomputer",
"host" => "LFR003256"
}

Do you know why ?

Thanks !

[purbon]

Actually you're right with this behaviour, for now the way it works is stripping out the configuration and then in the filter applying the set of operation in a given order, you can see it at https://github.com/logstash-plugins/logstash-filter-mutate/blob/master/lib/logstash/filters/mutate.rb#L211-L222 in case the configuration option was provided.

Changing this would require a certain design changes in logstash per se, but I see the pain of expecting the operation to be applied in a given order. We'll investigate more on how to provide an improvement for this behaviour. In the meanwhile if you aim to get a given order is good to have different mutates.

I would recommend for example your solution for now.

[mostolog]

Is this going to be solved (not a workaround using 2 blocks) in the near future?

[breml]

Maybe add the label docs as it would make sense to describe the defined order in the docs as in the docs the operations are listed in alphabetical order, which does not correspond to the order how the mutations are applied.