search

datahub-project / datahub

The Metadata Platform for your Data and AI Stack
https://datahubproject.io
Apache License 2.0
9.94k stars 2.94k forks source link

[DBT] Support Column Level Lineage on sources and seeds #10970

Closed ssassi closed 2 months ago

ssassi commented 4 months ago

Datahub is not currently supporting CLL on sources, seeds and tests as this code comment states.

CLL does work for these node types if the column names aren't changed on downstream entities, but that's not our case. Adding this support is required.

hsheth2 commented 3 months ago

@ssassi when might a source's column names differ from the columns on the warehouse table?

noel commented 2 months ago

I dont think this is an issue on sources and seeds. Dont think there is CLL on tests (or I dont know how to find it)

hsheth2 commented 2 months ago

@noel that's correct - we don't generate CLL for tests (which are modeled as DataHub assertions)

Will close this issue for now, but let me know if there's lingering issues with CLL for dbt.