NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
44 stars 34 forks source link

[FEA]Qualification tool to qualify a Dataproc Cluster instead of a single Spark Application #1074

Open viadea opened 4 weeks ago

viadea commented 4 weeks ago

Is your feature request related to a problem? Please describe. I wish Qualification tool can qualify a Dataproc Cluster instead of a single Spark Application.

Describe the solution you'd like Say there are 10 Spark Applications running in the same Dataproc Cluster, users need to decide if they want to migrate this Dataproc cluster from CPU to GPU. The qualification tool should look at all 10 Spark applications to make a cluster-level decision to see if this cluster is qualified or not.