MINOR -- Fix DQ Partition Issue - Githubissues

open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

https://open-metadata.org

Apache License 2.0

5.59k stars 1.05k forks source link

MINOR -- Fix DQ Partition Issue #18641

Open TeddyCr opened 1 week ago

TeddyCr commented 1 week ago

Describe your changes:

Fix data quality not taking partition settings into account
Remove handle_partition decorator
renamed self._sample to self._dataset and random_sample to get_dataset as contains either sampled, partitioned or raw dataset
Cleaned up datalake sampling retrieval
Added TABLESAMPLE support for POSTGRES percentage sampling

#

Type of change:

[x] Bug fix
[x] Improvement
[ ] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation

#

Checklist:

[x] I have read the CONTRIBUTING document.
[x] My PR title is Fixes <issue-number>: <short explanation>
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

Improvement

[x] I have added tests around the new logic.
[x] For connector/ingestion changes: I updated the documentation.