Is your feature request related to a problem? Please describe.
The stats table contains only entries of failed tests for the agg_dq and query_dq tests.
We see this as a limitation of the existing version for Spark Expectations, as no decent reporting can be built on:
How many tests are executed?
Is there progress in the context of DQ measurement? Do we see any percentage increase in successful tests?
Where all my tests successfully executed or did some test results not get written to the table due to some failures happening
Describe the solution you'd like
All results of the DQ tests being written to the stats table.
Additional context
Several users of Spark Expectations and especially those consuming the DQ results (stats table) are interested in this feature.
Is your feature request related to a problem? Please describe. The stats table contains only entries of failed tests for the agg_dq and query_dq tests. We see this as a limitation of the existing version for Spark Expectations, as no decent reporting can be built on:
Describe the solution you'd like
Additional context Several users of Spark Expectations and especially those consuming the DQ results (stats table) are interested in this feature.