awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
Apache License 2.0
3.32k stars 539 forks source link

Allow all DQ constraints to be generated from an Analyzer #508

Closed mentekid closed 1 year ago

mentekid commented 1 year ago

Issue #, if available:

Description of changes:

Due to our typing system, it is always possible to create an Analyzer, but converting that into a generic Constraint and generic Check is impossible.

Instead, give callers the option to convert an Analyzer into a Check with these helper functions. This enables reuse of Analyzers - you can use an Analyzer directly, or convert it to a Check by assigning it a severity and assertion.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.