search

open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
https://open-metadata.org
Apache License 2.0
4.77k stars 920 forks source link

A complet lineage using DBT manifest #15330

Open haithem-souala opened 4 months ago

haithem-souala commented 4 months ago

Is your feature request related to a problem? Please describe. We've an incomplet lineage because the data set integrated on OM, is hosting only obt, fact, dim tables.

Describe the solution you'd like

We've a bq dataset that hosts all models created by dbt (staging, inter, marts..) and another dataset that host only the marts (obt, fact, dim...) without the staging and inter models. I would like to expose only the seconde dataset on OM, but also having a complet lineage using dbt manifest.

Describe alternatives you've considered N/A

Additional context N/A

haithem-souala commented 4 months ago

This feature will also gives to OM user the possibility to ingest only the dbt project without the need to import the database service Metadata ( if the profiling is not needed).

haithem-souala commented 2 weeks ago

Hey @ulixius9 Any ETA?