logstash-plugins / logstash-output-elasticsearch_java

Java API Implementation of Elasticsearch Output
Apache License 2.0
0 stars 8 forks source link

Double always sent as string #36

Open MichaelDoyle opened 8 years ago

MichaelDoyle commented 8 years ago

I recently upgraded to Logstash 2.1.0 and Elasticsearch 2.1.0.

I'm trying to switch from the elasticsearch output (http) to the elasticsearch_java output (transport). Previously under Logstash 1.5 / Elasticsearch 1.7 we were using transport, and I'd like to do the same now.

I'm running into an issue where fields that were previously being indexed as double values (both with transport under LS 1.5/ ES 1.7, and with the http protocol under LS 2.1 / ES 2.1) are now being sent as string values using the elasticsearch_java output.

After switching from http to transport, we started seeing the following (We have coercion set to false in our index template):

MapperParsingException[failed to parse [test_double]]; nested: IllegalArgumentException[Double value passed as String];

This can be reproduced with a fairly trivial logstash config. We use log-courier, but the input shouldn't matter. The idea is to send some json data that contains double values.

e.g.)

_sample input_

{"message":"hello", "number": 1.2345}

_http_

number will be indexed as a double.

input {
  courier {
    port => "5041"
    ssl_certificate => ""
    ssl_key => ""
    ssl_verify => true
    ssl_verify_ca => ""
  }
}

filter {
  json {
    source => "message"
  }
}

output {
  elasticsearch {
    hosts => ["hostname:9200"]
    template => "/etc/logstash/templates/template.json"
    template_overwrite => true
  }

  stdout { codec => rubydebug }
}

_transport_

number will be sent as string.

input {
  courier {
    port => "5041"
    ssl_certificate => ""
    ssl_key => ""
    ssl_verify => true
    ssl_verify_ca => ""
  }
}

filter {
  json {
    source => "message"
  }
}

output {
  elasticsearch_java {
    hosts => ["hostname:9300"]
    network_host => "0.0.0.0"
    cluster => "logstash"
    template => "/etc/logstash/templates/template.json"
    template_overwrite => true
  }

  stdout { codec => rubydebug }
}

Happy to provide more details. Let me know if this is not the right place to report this.