dlt-hub / dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
https://dlthub.com/docs
Apache License 2.0
2.65k stars 176 forks source link

[Experiment] Basic column lineage with sqlglot #2065

Open sh-rp opened 2 days ago

sh-rp commented 2 days ago

Description

Implements a few helper functions that, given a simplified schema and an sql statement, can determine what the names of the resulting columns are and from which original table and column they came from. We can use this together with our dlt schema to compute the dlt schema of a dataset query ahead of time and set the right hints.

Notes:

netlify[bot] commented 2 days ago

Deploy Preview for dlt-hub-docs canceled.

Name Link
Latest commit 5ea49ee5f038dce71b4c69f23d6c7dc2bc87d690
Latest deploy log https://app.netlify.com/sites/dlt-hub-docs/deploys/6736576d2decf20008ed656f