AICoE / prometheus-anomaly-detector

A newer more updated version of the prometheus anomaly detector (https://github.com/AICoE/prometheus-anomaly-detector-legacy)
GNU General Public License v3.0
597 stars 151 forks source link

test_model should log number of anomalies in data set to MLflow #32

Closed MichaelClifford closed 5 years ago

MichaelClifford commented 5 years ago

test_model.py should also log the total number of ground truth anomalies as defined by the auto-labeling procedure, the total number of forecasted anomalies, and the total number of data points for a given test set to MLflow. This will give the True positive rate value more context when looking at the result of different testing runs.

See the output of cell 5 in the notebook POC for an example: https://github.com/AICoE/prometheus-anomaly-detector/blob/master/notebooks/prometheus_anomaly_detection_test.ipynb

hemajv commented 5 years ago

I tried running the test for a metric which was anomalous according to the thoth team and wanted to cross-verify if the Prophet model predicted it as an anomaly as well. Logging this information would be useful