search

NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
44 stars 34 forks source link

[FEA] Add option to switch between Databricks configuration profiles in Qual tool #1061

Open cindyyuanjiang opened 1 month ago

cindyyuanjiang commented 1 month ago

I wish Qual tool can support switching between different Databricks configuration profiles.

E.g. I have a Databricks configuration profile for DB AWS and one for DB Azure. Each time I want to run Qual tool on a different workspace, I have to re-configure my DEFAULT profile.

Ref: https://docs.databricks.com/en/dev-tools/auth/index.html#config-profiles

amahussein commented 1 month ago

I remember we have --profile option in spark_rapids_user_tools cmd, right?

cindyyuanjiang commented 1 month ago

I remember we have --profile option in spark_rapids_user_tools cmd, right?

Yes, we do, but not in spark_rapids CLI.

amahussein commented 1 month ago

I see. Is this going to be part of the usability improvements? The dilemma with spark_rapids is that --platform is optional. This means that we have to add support of --profile to all the CSPs