Nike-Inc / spark-expectations

A Python Library to support running data quality rules while the spark job is running⚡
https://engineering.nike.com/spark-expectations
Apache License 2.0
161 stars 37 forks source link

Adding range comparision functionality to aggdq Detailed Stats table #86

Closed nishantsingh93 closed 5 months ago

nishantsingh93 commented 5 months ago

Description

This change introduces range comparison for Aggregation Data Quality expectation. A user can now provide valid SQL expression to compare upper_bound and lower_bound check against a measure

Related Issue

https://github.com/Nike-Inc/spark-expectations/issues/81

Motivation and Context

This feature request is to enhance the Aggregation Data quality to accept a range expression. There are two types of valid SQL expression that are supported.

Types of changes

Checklist:

codecov[bot] commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (19ed71a) to head (5faef7f).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #86 +/- ## =========================================== + Coverage 0 100.00% +100.00% =========================================== Files 0 22 +22 Lines 0 1837 +1837 =========================================== + Hits 0 1837 +1837 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.