search

maropu / spark-sql-flow-plugin

Visualize column-level data lineage in Spark SQL
Apache License 2.0
86 stars 15 forks source link

SQLContractedFlow has no planToSQLFlow method #4

Closed melin closed 2 years ago

melin commented 2 years ago

SQLContractedFlow has no planToSQLFlow method，How to obtain the compressed Lineage data directly from optimizedPlan?

maropu commented 2 years ago

Yea, nice catch. I'll work on this.

maropu commented 2 years ago

Fixed in https://github.com/maropu/spark-sql-flow-plugin/commit/b5c1b6e57585d353bd2ed32da4014e551eb23ce9

maropu commented 2 years ago

https://github.com/maropu/spark-sql-flow-plugin/blob/b5c1b6e57585d353bd2ed32da4014e551eb23ce9/src/main/scala/org/apache/spark/sql/SQLFlow.scala#L803-L820