search

JahstreetOrg / spark-on-kubernetes-helm

Spark on Kubernetes infrastructure Helm charts repo
Apache License 2.0
198 stars 76 forks source link

increase k8s upper bound #50

Closed fsanaulla closed 3 years ago

fsanaulla commented 3 years ago

k8s upper bound was increased for livy and spark

jahstreet commented 3 years ago

Hi. Thank you for the PR. Currently the default images are built with Spark v3.0.1, which works with Fabric8 K8s client v4.9.2. This version is compatible with K8s APIs up to v1.18.0 (exclusive). So the fact that it works with 1.18.9 is just a luck. I would prefer not to extend the support to 1.19.x before Spark releases the version with the upgraded Fabric8 K8s client to v4.13.0+. In the meantime I can suggest you to fork the project and provide this patch for your environment only. Best.

fsanaulla commented 3 years ago

Good point, thank you!