spark big query jar only works with scala 2.11

kubeflow / spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Apache License 2.0

2.81k stars 1.38k forks source link

spark big query jar only works with scala 2.11 #1225

Closed suvashishtha closed 1 month ago

suvashishtha commented 3 years ago

I am using gcr.io/spark-operator/spark:v3.0.0-gcs-prometheus and running my job in GKE cluster. When I upgraded scala 2.12, the job started giving me ClasscastException about version incompatibility between scala versions 2.11 & 2.12 (I do not have exact error as I fixed it already )

To fix it, i had to remove spark-bigquery-latest.jar and add spark-bigquery_2.12-0.19.1.jar to docker image

suvashishtha commented 3 years ago

@liyinan926 any thoughts?

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 1 month ago

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.