[BUG] Handle scenarios where CSP CLI is unavailable

NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS

Apache License 2.0

49 stars 35 forks source link

[BUG] Handle scenarios where CSP CLI is unavailable #988

Closed parthosa closed 3 weeks ago

parthosa commented 4 months ago

We should handle the cases when users do not have CSP CLI installed (eg aws, az, gcloud). The existing CLI calls are mainly for pulling instance type descriptions. We can cache these information to remove dependency on the CLI.

### Tasks
- [ ] https://github.com/NVIDIA/spark-rapids-tools/issues/1123
- [ ] https://github.com/NVIDIA/spark-rapids-tools/issues/910
- [ ] https://github.com/NVIDIA/spark-rapids-tools/issues/1191
- [ ] https://github.com/NVIDIA/spark-rapids-tools/issues/1238

kuhushukla commented 4 months ago

Additionally without the cli for local eventlog location , using platform flag for a CSP can throw the following error.

File "/home/kuhu/py310/lib/python3.10/site-packages/spark_rapids_pytools/common/utilities.py", line 333, in exec
    raise RuntimeError(f'{cmd_err_msg}')
RuntimeError: Error invoking CMD <aws ec2 describe-instance-types --region None --instance-types r5d.4xlarge>:
        | /bin/bash: line 1: aws: command not found

parthosa commented 3 months ago

Unassigning this as we need to come up with plan to cache instance descriptions and remove dependency on CSP CLIs