Gauge mapping errors

[nilsga]

When the dynamic mapping has created the mapping for a gauge of type T, other gauges of other types fails with mapping errors.

[nilsga]

One solution might be to have a Gauge-mapping for each data type.

[spinscale]

do you have an example I could use for a failing test here?

[Nycander]

I stumbled upon this bug while tryingon elasticsearch-metrics-reporter in a http://www.dropwizard.io installation. Seems to be quite common for gauges with different value types.

Tried to create a failing test here, sometimes there's an exception on the output, but I failed to reproduce it consistently and assert on it since the exception happens inside elasticsearch. https://github.com/Nycander/elasticsearch-metrics-reporter-java/commit/5f12d4f011ddff7dd0a2ffa00a8cde84188ff94b#diff-fc3406b0dc3bcd74bc9935e07f6b862fR242

Anyway, when the problem is that elasticsearch automatically creates a mapping for the first value, which is a double:

  "gauge" : {
    "_all" : {
      "enabled" : false
    },
    "properties" : {
      "@timestamp" : {
        "type" : "date",
        "format" : "dateOptionalTime"
      },
      "name" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "value" : {
        "type" : "double"
      }
    }
  }

When lateron a gauge comes along with a string value, the following exception is thrown in elasticsearch:

[2015-10-11 11:04:26,572][DEBUG][action.bulk              ] [Maur-Konn] [metrics-2015-10][0] failed to execute bulk item (index) index {[metrics-2015-10][gauge][AVBWJNfILvCCGXSkD87d], source[{"name":"jvm.attribute.vendor","@timestamp":"2015-10-11T09:04:26.000+0000","value":"Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.51-b03 (1.8)"}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [value]
    at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:409)
    at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
    at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
    at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:453)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NumberFormatException: For input string: "Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.51-b03 (1.8)"
    at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
    at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
    at java.lang.Double.parseDouble(Double.java:538)
    at org.elasticsearch.common.xcontent.support.AbstractXContentParser.doubleValue(AbstractXContentParser.java:182)
    at org.elasticsearch.index.mapper.core.DoubleFieldMapper.innerParseCreateField(DoubleFieldMapper.java:310)
    at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:236)
    at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:399)
    ... 12 more

and the gauge is lost :(

[davidkarlsen]

Stagemonitor seems to use an approach where the value is mapped to a given type: https://www.elastic.co/blog/elasticsearch-as-a-time-series-data-store (value, value_string, value_boolean)

[static-max]

I created a PR for this: #49