awslabs / python-deequ

Python API for Deequ
Apache License 2.0
669 stars 131 forks source link

Is there a way to map the results(constraint status) of VerificationSuite's run() method to the metrics stored in metrics repository? #145

Open sharadcapillary opened 11 months ago

sharadcapillary commented 11 months ago

Discussed in https://github.com/awslabs/python-deequ/discussions/141

Originally posted by **sharadcapillary** July 19, 2023 Hi, I was building a system over deequ, where ppl can write test cases on data using deequ and save the run metrics for visualization and later anomaly detection. However the metrics that get saved in the metrics repository by default don't contain the constraint status i.e., whether a constraint passed or failed. It just holds the value. I want my system to be automated, and I want to store the metrics and the run status for each constraint. 1. So is there any way to map the result of the .run() command to each row in the metrics stored in the metrics repository by deequ? 2. Also Is there a way to save different Result Key w.r.t each constraint for the same "Check" object? I am aware that I can achieve the above using different "Check" objects for each constraint, however that would be very inefficient I believe. Image shows the result of VerificationSuite.run() image Image shows the result of loading the metrics from metrics repository image