holdenk / spark-flowchart

Flowchart for debugging Spark applications
100 stars 27 forks source link

How to compare two event logs #24

Closed attilapiros closed 2 years ago

attilapiros commented 2 years ago

The idea to load stage metrics into a view is coming from @LucaCanali (I have seen it in his notes Spark EventLog). Thank you @LucaCanali!

I have just extended it with a join to calculate ratio between stage metrics coming from the two different event logs within the same stage.

I hope you will find this useful. Comments are welcomed!

@holdenk @atbida Thanks for the presentation and for the flowchart.