nathanielc / morgoth

Metric anomaly detection
http://docs.morgoth.io
Apache License 2.0
280 stars 31 forks source link

When to use the different fingerprints or a combination of the #43

Closed johankcarlsson closed 7 years ago

johankcarlsson commented 7 years ago

Hi, could you elaborate a bit on when to use the different fingerprints and if it always best to combine them for consensus. How does a tick-script look when several fingerprints are used.

nathanielc commented 7 years ago

The purpose of the other fingerprints is to work with data that is not normally distributed. In practice I have found that the sigma fingerprint is almost always sufficient. Next in line would be the jsdiv fingerprinter, but I have found that the kstest is a little to sensitive and often causes more harm than good.

In short experiment with them and see if they improve your desired results.

Example using all fingerprinters

  @morgoth()
        // track the 'usage_idle' field
        .field('usage_idle')
        .errorTolerance(0.01)
        // The window is anomalous if it occurs less the 5% of the time.
        .minSupport(0.05)
        // Configuration for sigma
        // deviations: Number of stddevs away the mean must be to count as a new fingerprint.
        .sigma(deviations)

        // Configuration for jsdiv
        //  min: Excpected minimum value of the window data.
        //  max: Excpected maximum value of the window data.
        //  binwidth: Size of a bin for the histogram
        //  pValue: Standard p-value statistical threshold. Typical value is 0.05
        .jsdiv(min, max, binWidth,pValue)

         // Configuration for kstest
         //  The only parameter is a confidence level.
         //  Valid values are from 0-5.
         //  The level maps to a list of predefined critical values for the KS test.
         //  Increasing 'confidence' decreases the number of anomalies detected.
         .kstest(confidence)