awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
Apache License 2.0
3.27k stars 536 forks source link

Support for Custom SQL Execution in Deequ Library #551

Open skarthikbigdata opened 6 months ago

skarthikbigdata commented 6 months ago

Is your feature request related to a problem? Please describe. I would like to request support for executing custom SQL queries in the Deequ library. This would allow users to define and execute custom data quality checks and verifications using SQL statements.

Describe the solution you'd like As a user of the Deequ library, I often need to perform complex data quality checks that are not supported out of the box. Being able to write and execute custom SQL queries would greatly enhance the flexibility and usability of the library for my use case.

Describe alternatives you've considered