search

elementary-data / elementary

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
https://www.elementary-data.com/
Apache License 2.0
1.87k stars 158 forks source link

Not able to set up detection period - 'hour' for column_anomalies tests #1646

Open ZuzanaRaichlova opened 1 month ago

ZuzanaRaichlova commented 1 month ago

I'm trying to set up the detection period for our column_anomalies tests to hour (we are missing a lot of alerts because of the 1 day setting). But I'm getting this error: Missing mandatory configuration: ['backfill_days'] I saw a couple of people had this issue earlier this year. Is there already a solution for it? Because in the docs I can see that hour should be a supported period. Thank you!

To Reproduce

detection_period: period: hour count: 2

-> runing the test -> Compilation error Missing mandatory configuration: ['backfill_days']

Environment: Elementary package version: 0.15.2 dbt version: 1.2.0 Data warehouse: snowflake

dapollak commented 1 month ago

Hey @ZuzanaRaichlova, It does support, but unfortunately it must be set to a value larger than 24. (detection period must be longer than one day)

ZuzanaRaichlova commented 1 month ago

Thank you for the quick response! Are there any plans to allow also shorter periods?

annav00 commented 1 month ago

I'm also interested in this question