NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
822 stars 235 forks source link

Let AWS Databricks automatically choose an Availability Zone #11714

Closed NvTimLiu closed 1 week ago

NvTimLiu commented 1 week ago

To address the AWS Databricks failure: InsufficientInstanceCapacity: Could not complete request as AWS does not currently have enough available capacity to fulfill your request for instance type g5.4xlarge.

Our system will be working on provisioning additional capacity. You can currently get g5.4xlarge capacity by not specifying an Availability Zone in your request or choosing us-west-2a, us-west-2b.

Also, print the cluster ID to stdout so that a calling script can easily access it

NvTimLiu commented 1 week ago

build