awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
Apache License 2.0
3.32k stars 539 forks source link

Using anomaly detection for custom non-data metrics #429

Open avanderm opened 2 years ago

avanderm commented 2 years ago

Hey, I really like the anomaly detection options of Deequ and would like to ask if they can be used for metrics outside of Deequ? Some examples:

Are there any options to include some non-data intensive metrics and perhaps even add them to the metrics repository (need for anomaly detection after all)?

Kind regards,

Antoine