open-metadata / openmetadata-spark-agent

Apache License 2.0
2 stars 2 forks source link

Improve column level lineage #13

Open ulixius9 opened 3 months ago

ulixius9 commented 3 months ago
spark.sql("insert into table1 (col1) select concat(col2, col3) from table2 limit 100").show()

this query only generates column level lineage between col1 and col2, we are missing col3 in this lineage