GEN-970: Refactor redshift system metrics to support freshness test

open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

https://open-metadata.org

Apache License 2.0

5.26k stars 994 forks source link

GEN-970: Refactor redshift system metrics to support freshness test #17981

Open sushi30 opened 3 days ago

sushi30 commented 3 days ago

Describe your changes:

moved redshift system metrics to the redshift source module
use Timestamp in data quality
added plugin feature to test utils

[ ] run e2e tests

E2E runs

https://github.com/open-metadata/OpenMetadata/actions/workflows/py-cli-e2e-tests.yml?query=branch%3Aredshift-system-metrics-refactor

Type of change:

[ ] Bug fix
[ ] Improvement
[ ] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] My PR title is Fixes <issue-number>: <short explanation>
[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

notion-workspace[bot] commented 3 days ago

Freshness test case by system metrics

github-actions[bot] commented 2 days ago

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR. You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

sonarcloud[bot] commented 19 hours ago

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
47.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions[bot] commented 19 hours ago

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR. You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions[bot] commented 14 hours ago

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR. You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.