Closed cindyyuanjiang closed 3 months ago
Currently, DB AWS qual tool extracts pricing information from https://github.com/NVIDIA/spark-rapids-tools/blob/dev/user_tools/src/spark_rapids_pytools/resources/databricks-premium-catalog.json. However, this file could be outdated and has not been actively maintained.
We recently found Databricks saves AWS pricing info under: https://www.databricks.com/en-website-assets/data/pricing/AWS.json. Qual tool should utilize this website to maintain DB AWS pricing info.
Currently, DB AWS qual tool extracts pricing information from https://github.com/NVIDIA/spark-rapids-tools/blob/dev/user_tools/src/spark_rapids_pytools/resources/databricks-premium-catalog.json. However, this file could be outdated and has not been actively maintained.
We recently found Databricks saves AWS pricing info under: https://www.databricks.com/en-website-assets/data/pricing/AWS.json. Qual tool should utilize this website to maintain DB AWS pricing info.