Skip SparkTableMetricsRepositoryTest test when SupportsRowLevelOperations is not available

awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

Apache License 2.0

3.18k stars 519 forks source link

Skip SparkTableMetricsRepositoryTest test when SupportsRowLevelOperations is not available #536

Closed eycho-am closed 4 months ago

eycho-am commented 4 months ago

Issue #, if available:

Description of changes: The SparkTableMetricRepositoryTest file was added in this PR #518. The test save and load to iceberg a single metric uses the SupportsRowLevelOperations class which is supported from spark 3.3.

This change modifies the test so we skip it for spark versions lower than 3.3

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.