NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
821 stars 235 forks source link

[FEA] Explain API/Qualification improvements #5177

Open tgravescs opened 2 years ago

tgravescs commented 2 years ago

Is your feature request related to a problem? Please describe. Currently the explain api just dumps out text of what can run on GPU and what won't. It would be nice to aggregate that information together to an easier readable format. Also if possible include other metrics and data from qualification tool to really give the user idea if this is a good candidate without them having to manually combine them.

revans2 commented 2 years ago

I totally agree. I did a quick hack with this type of thing and it worked out okay. The biggest problem is that the plan when we get work on it has not gone through the exchange reuse optimization yet. That means that we might end up counting some paths multiple times in any roll-up metrics.