search

open-metadata / openmetadata-spark-agent

Apache License 2.0
2 stars 2 forks source link

Data lineage between different database services is not showing #11

Open abhilashaam opened 4 months ago

abhilashaam commented 4 months ago

We have openmetadata setup on AKS clsuter using helm chart. we have connected databricks and azure sql database services. when we create table on databricks using a table from azure sql, the data lineage should be azure_sql_table --> databricks_table, but this lineage is not coming, we can see lineage between tables on databricks, like databricls_table1 --> databricks_table2. we tried creating tables with openmetadata-spark-agent as well as openmetadata-spark-agent-1.0-beta, nothing is giving expected result.

abhilashaam commented 4 months ago

Hi team, Any update on this issue?

Madhur-Jatiya commented 2 months ago

Is lineage btwn 2 different tables in the single database is possible?

ulixius9 commented 1 month ago

@Madhur-Jatiya yes that is possible