Mismatched Keys: Update read_dlt_delta() with key "source_database" instead of "database"

databrickslabs / dlt-meta

This is metadata driven DLT based framework for bronze/silver pipelines

Other

125 stars 54 forks source link

Mismatched Keys: Update read_dlt_delta() with key "source_database" instead of "database" #33

Closed rtdtwo closed 2 months ago

rtdtwo commented 5 months ago

Issue:

When the bronze_dataflow_spec dataframe is created in onboard_dataflowspec.py, the database is included with key source_database in the sourceDetails field. However, in pipeline_readers.py > read_dlt_delta(), it is accessed with key database instead.

Fix:

Change the key to source_database in pipeline_readers.py > read_dlt_delta().

CLAassistant commented 5 months ago

All committers have signed the CLA.

ravi-databricks commented 5 months ago

Please fix Unit Tests

prabhat-ratnala commented 2 months ago

Can this be resolved please?

ravi-databricks commented 2 months ago

merged this for v0.0.7 release.

ravi-databricks commented 2 months ago

@prabhat-ratnala this is fixed in release v.0.0.7