nathanielc / morgoth

Metric anomaly detection
http://docs.morgoth.io
Apache License 2.0
280 stars 31 forks source link

Task stops with error: failed to register metrics #54

Closed NNMavy closed 7 years ago

NNMavy commented 7 years ago

When using morgoth version 0.3.0 or above kapacitor tasks stop running with the error:

Stopped task: cpu_alert morgoth3: failed to register metrics for group: "cpu=cpu-total,host=host01": window count metric: duplicate metrics collector registration attempted

Kapacitor is version 1.3.2 and Influx is version 1.3.3. Morgoth seems to be working ok until commit 63efe5f8324c3ff17066867363c0431b9a08ecfb, anything after that fails with the above mentioned error.

The tick script used for testing is:

var measurement = 'cpu'
var groups = [*]
var whereFilter = lambda: "cpu" == 'cpu-total'
var window = 1m
var field = 'usage_idle'

var scoreField = 'anomalyScore'
var minSupport = 0.05
var errorTolerance = 0.01
var consensus = 0.5

// Number of sigmas allowed for normal window deviation
var sigmas = 3.0

stream
  // Select the data we want
  |from()
      .database('telegraf')
      .retentionPolicy('autogen')
      .measurement(measurement)
      .groupBy(groups)
      .where(whereFilter)
  // Window the data for a certain amount of time
  |window()
     .period(window)
     .every(window)
     .align()
  // Send each window to Morgoth
  @morgoth()
     .field(field)
     .scoreField(scoreField)
     .minSupport(minSupport)
     .errorTolerance(errorTolerance)
     .consensus(consensus)
     // Configure a single Sigma fingerprinter
     .sigma(sigmas)
  // Morgoth returns any anomalous windows
  |alert()
     .details('')
     .crit(lambda: TRUE)
     .log('/tmp/cpu_alert.log')