Question: anomaly not directly related to thresholds

[daradermody]

I'm seeing the following results from the prediction/anomaly calculations:

Expand

``` { "id": "985fd8e5-5a15-4876-820a-4df9e9f1bc0f", "result": { "observed": { "value": [ -0.6180654168128967, -0.7154610753059387, -0.8006424903869629 ] }, "predicted": { "lower_value": [ -0.6362288721576379, -0.7298990019336389, -0.8107846506610559 ], "upper_value": [ -0.6246791354641272, -0.7183492652401282, -0.7992349139675452 ], "value": [ -0.6304540038108826, -0.7241241335868835, -0.8050097823143005 ] }, "stats": [ { "anomalies": {}, "anomaly": false, "mse": 0.00008253296845476636, "score": 0.6196698904133657 }, { "anomalies": {}, "anomaly": false, "mse": 0.00008253296845476636, "score": 0.4333211185803876 }, { "anomalies": {}, "anomaly": false, "mse": 0.00008253296845476636, "score": 0.21844939301443045 } ], "timestamps": [ 1539951505.0, 1539951510.0, 1539951515.0 ] }, "state": "done", "type": "prediction" } ```

The observed values are below the lower thresholds, but it is not considered an anomaly. Is there something I'm missing? What is the minimum value of score that categorises it as an anomaly?

[regel]

Hi @daradermody, what are your min_threshold and max_threshold settings?

Tip: we've introduced in 1.4 the ability to set min_threshold=0 and max_threshold=0 to let loudmld calculate these values.

The hysteresis goes like this during the anomaly detection:

• New anomaly starts if score >= max_threshold
• Anomaly ends if the score < min_threshold

1.4.2 in more detail:

• Scores are calculated either using the L1 norm (the default), or a gaussian distribution.
• lower* and upper* have first been introduced to display graphs, and use 68–95–99.7 empirical rule based on the training MSE to calculate an interval.

[daradermody]

For the example I gave, I didn't set min_threshold and max_threshold, but they seem to default to 0. I guess I incorrectly assumed that the upper and lower values were thresholds rather than just the deviation of the source signal. Are they calculated as one standard deviation from the predicted?

[regel]

Will be more consistent in 1.5 release: min_threshold and maxthreshold as well as lower upper_ will all use the 68–95–99.7 rule. It's easier to interpret.

We no longer use the reconstruction error (mse), but instead use the reconstruction probability metric.

[regel]

fixed in 1.5.0