Add lineage details for other entities

open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

https://open-metadata.org

Apache License 2.0

5.44k stars 1.03k forks source link

Add lineage details for other entities #9281

Closed pmbrull closed 6 months ago

pmbrull commented 1 year ago

Currently, we are using lineage details to pass col-level lineage. Let's update the implementation on the UI (and possibly backend/ingestion) to handle:

task-level lineage for Pipelines [maybe not, since pipelines are edge now]
chart-level lineage for dashboards
feature-level lineage for ML Models
field-level lineage for topics
dashboard datamodel

pmbrull commented 1 year ago

Re Pipelines & tasks: https://openmetadata.slack.com/archives/C02B6955S4S/p1684146703741239

 i am suggesting like this :- Tasks <> Table  instead of Pipeline <> Table (which is the case currently ) .
Though this might be very specific usecase. but it helps as when different tasks are working with different tables inside same pipeline.

dipayan80 commented 11 months ago

We would like to have column level lineage for the tableau dashboards

harshsoni2024 commented 6 months ago

I will work on this.

VarshaNagpal commented 6 months ago

Hi Open Metadata team,

Could you please enable the column level lineage for Power BI dashboards to Snowflake columns ? And please let us know the ETA for the same.

Regards, Varsha