databrickslabs / dlt-meta

This is metadata driven DLT based framework for bronze/silver pipelines
Other
125 stars 54 forks source link

37 migrate to create streaming table api from create streaming live table #39

Closed ravi-databricks closed 3 months ago

ravi-databricks commented 3 months ago
  1. Upgraded dlt api from create_streaming_live_table to latest create_streaming_table
  2. Added data quality expectations for silver since create_streaming_table api takes expectations as arguments
  3. Added unit tests
  4. Corrected demo's and integration tests to incorporate silver_data_quality_expectations
codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 82.35294% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 89.90%. Comparing base (1ff4b45) to head (965d075). Report is 2 commits behind head on main.

Files Patch % Lines
src/dataflow_pipeline.py 79.16% 2 Missing and 3 partials :warning:
src/onboard_dataflowspec.py 88.88% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #39 +/- ## ========================================== - Coverage 90.28% 89.90% -0.39% ========================================== Files 8 8 Lines 803 832 +29 Branches 149 161 +12 ========================================== + Hits 725 748 +23 - Misses 31 33 +2 - Partials 47 51 +4 ``` | [Flag](https://app.codecov.io/gh/databrickslabs/dlt-meta/pull/39/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=databrickslabs) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/databrickslabs/dlt-meta/pull/39/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=databrickslabs) | `89.90% <82.35%> (-0.39%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=databrickslabs#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.