Closed MichaelClifford closed 5 years ago
I tried running the test for a metric which was anomalous according to the thoth team and wanted to cross-verify if the Prophet model predicted it as an anomaly as well. Logging this information would be useful
test_model.py
should also log the total number of ground truth anomalies as defined by the auto-labeling procedure, the total number of forecasted anomalies, and the total number of data points for a given test set to MLflow. This will give theTrue positive rate
value more context when looking at the result of different testing runs.See the output of cell 5 in the notebook POC for an example: https://github.com/AICoE/prometheus-anomaly-detector/blob/master/notebooks/prometheus_anomaly_detection_test.ipynb