ccin2p3 / samplerr

Round robin timeseries middleware based on riemann and elasticsearch
Eclipse Public License 1.0
15 stars 4 forks source link

Duplicate documents indexed #7

Closed faxm0dem closed 8 years ago

faxm0dem commented 8 years ago

It seems multiple documents are being indexed with the same (time cfunc step) but with different metric:

☠ curl '0:9200/.samplerr-2016.03.18/_search?pretty' -d '{"query": {"filtered": {"query": {"bool": {"should": [{"query_string": {"query": "plugin:load AND cfunc:min AND step:20"}}]}},"filter"
: {"bool": { "must": [{"range": {"@timestamp": {"from": 1458293631000,"to": 1458293631000}}}]}}}}}'
{
  "took" : 16,
  "timed_out" : false,
  "_shards" : {
    "total" : 18,
    "successful" : 18,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 6.336565,
    "hits" : [ {
      "_index" : ".samplerr-2016.03.18",
      "_type" : "samplerr",
      "_id" : "AVOJEvnGS5-_C1ONdglq",
      "_score" : 6.336565,
      "_source":{"smurf_machine_type":"Virtual KVM","plugin":"load","host":"ccswiss","ds_index":"2","tags":["usag.riemann_collectd","status.Up","collectd"],"tf":"YYYY.MM.dd","ds_name":"longterm","smurf_status":"Up","@timestamp":"2016-03-18T09:33:51Z","type":"load","metric":3.58,"state":"ok","ds_type":"gauge","cfunc":"min","smurf_rack":"osservice1","service":"load/load/longterm/min/20","smurf_usage":"riemann_collectd","step":20}
    }, {
      "_index" : ".samplerr-2016.03.18",
      "_type" : "samplerr",
      "_id" : "AVOJEvnGS5-_C1ONdglk",
      "_score" : 6.3113976,
      "_source":{"smurf_machine_type":"Virtual KVM","plugin":"load","host":"ccswiss","ds_index":"1","tags":["usag.riemann_collectd","status.Up","collectd"],"tf":"YYYY.MM.dd","ds_name":"midterm","smurf_status":"Up","@timestamp":"2016-03-18T09:33:51Z","type":"load","metric":3.35,"state":"ok","ds_type":"gauge","cfunc":"min","smurf_rack":"osservice1","service":"load/load/midterm/min/20","smurf_usage":"riemann_collectd","step":20}
    }, {
      "_index" : ".samplerr-2016.03.18",
      "_type" : "samplerr",
      "_id" : "AVOJEvnGS5-_C1ONdglS",
      "_score" : 6.237317,
      "_source":{"smurf_machine_type":"Virtual KVM","plugin":"load","host":"ccswiss","ds_index":"0","tags":["usag.riemann_collectd","status.Up","collectd"],"tf":"YYYY.MM.dd","ds_name":"shortterm","smurf_status":"Up","@timestamp":"2016-03-18T09:33:51Z","type":"load","metric":0.93,"state":"ok","ds_type":"gauge","cfunc":"min","smurf_rack":"osservice1","service":"load/load/shortterm/min/20","smurf_usage":"riemann_collectd","step":20}
    } ]
  }
}
faxm0dem commented 8 years ago

PBCAK: load has three values I hate multi-ds values :)