[TASK] cudf dropped python 3.9 support

[pxLi]

Describe the bug as https://docs.rapids.ai/notices rapids is going to drop python 3.9 support, so all of our related usage require an update

https://github.com/NVIDIA/spark-rapids/blob/branch-24.10/jenkins/Dockerfile-blossom.integration.rocky#L54 https://github.com/NVIDIA/spark-rapids/blob/branch-24.10/jenkins/Dockerfile-blossom.integration.ubuntu#L66

and internal cudf_udf pipelines (databricks + standalone).

lets try make all the defaults to 3.10 for non-CSP environments, and for databricks test cudf_udf only when the default python is supported by rapids

NOTE: if the original environment does not rely on cudf-py (conda,pip), we should keep the python version as pyspark should support all 3.8 and above

Steps/Code to reproduce bug Please provide a list of steps or a code sample to reproduce the issue. Avoid posting private or sensitive data.

Expected behavior A clear and concise description of what you expected to happen.

Environment details (please complete the following information)

• Environment location: [Standalone, YARN, Kubernetes, Cloud(specify cloud provider)]
• Spark configuration settings related to the issue

Additional context Add any other context about the problem here.

[pxLi]

keep this open to verify cudf_udf cases on other databricks runtimes