elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.18k stars 3.5k forks source link

Java execution can't compare floating points and longs in conditionals #12128

Open jsvd opened 4 years ago

jsvd commented 4 years ago
input {
  generator {
    lines => [
      '{"number": 10}',
      '{"number": 10.0}'
    ]
    count => 1
    codec => json
  }
}
filter {
  if [number] == 10 {
    mutate { add_tag => ['same_long'] }
  }
  if [number] == 10.0 {
    mutate { add_tag => ['same_float'] }
  }
}
output {
  stdout { }
}

This results in:

{
      "@version" => "1",
      "sequence" => 0,
        "number" => 10.0,
          "host" => "joaos-mbp.lan",
    "@timestamp" => 2020-07-21T13:40:03.590Z,
          "tags" => [
        [0] "same_float"
    ]
}
{
      "@version" => "1",
      "sequence" => 0,
        "number" => 10,
          "host" => "joaos-mbp.lan",
    "@timestamp" => 2020-07-21T13:40:03.579Z,
          "tags" => [
        [0] "same_long"
    ]
}

While in 6.8 (ruby execution) resulted in:

{
    "@timestamp" => 2020-07-21T13:43:13.878Z,
      "@version" => "1",
          "host" => "joaos-mbp.lan",
      "sequence" => 0,
        "number" => 10,
          "tags" => [
        [0] "same_long",
        [1] "same_float"
    ]
}
{
    "@timestamp" => 2020-07-21T13:43:13.889Z,
      "@version" => "1",
          "host" => "joaos-mbp.lan",
      "sequence" => 0,
        "number" => 10.0,
          "tags" => [
        [0] "same_long",
        [1] "same_float"
    ]
}

The current workaround is to rely on mutate { convert { "field1" => "float" }} to ensure both sides are floating point.

jsvd commented 3 years ago

Adding a note here that fixing this will still allow wrong comparisons to happen, e.g.:

irb(main):003:0> 1.00000000000000001 > 1
=> false

But since using the mutate filter to coerce the int to float doesn’t fix this anyway:

irb(main):006:0> 1.00000000000000001 > 1.0
=> false

Labelling this as a bug, as fixing this will make a lot more comparisons be truthful, and won't break any existing truthful comparison.

Comparing floats is always a pain. A way to improve the correctness of these comparisons is to change the float1 == float2 test to a float1 - float2 < X or similar.