databrickslabs / dlt-meta

This is metadata driven DLT based framework for bronze/silver pipelines
Other
125 stars 54 forks source link

Allow to specify source database instead of defaulting to bronze database in __get_silver_dataflow_spec_dataframe #34

Open rtdtwo opened 5 months ago

rtdtwo commented 5 months ago

Issue

The FAQs describes that any data source can be used for silver tables. However, the code always tries to find the bronze database from the onboarding rows, and there's no explicit mention of a source database other than a bronze database.

Fix

Change the code to accomplish the following:

codecov[bot] commented 5 months ago

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (2a93dd9) 90.28% compared to head (f4fbd2b) 89.95%.

Files Patch % Lines
src/onboard_dataflowspec.py 40.00% 1 Missing and 2 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #34 +/- ## ========================================== - Coverage 90.28% 89.95% -0.34% ========================================== Files 8 8 Lines 803 806 +3 Branches 149 151 +2 ========================================== Hits 725 725 - Misses 31 32 +1 - Partials 47 49 +2 ``` | [Flag](https://app.codecov.io/gh/databrickslabs/dlt-meta/pull/34/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=databrickslabs) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/databrickslabs/dlt-meta/pull/34/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=databrickslabs) | `89.95% <40.00%> (-0.34%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=databrickslabs#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

ravi-databricks commented 5 months ago

@rtdtwo, Needs to add unit tests for added new lines as per above code coverage report.