jupyter-sparkmagic chart uses k8s APIs Removed In 1.16

JahstreetOrg / spark-on-kubernetes-helm

Spark on Kubernetes infrastructure Helm charts repo

Apache License 2.0

198 stars 76 forks source link

jupyter-sparkmagic chart uses k8s APIs Removed In 1.16 #38

Closed hynix closed 3 years ago

hynix commented 3 years ago

Cannot install jupyter-sparkmagic 1.0.0 chart on k8s 1.16+ Error occurred: no matches for kind "Deployment" in version "apps/v1beta2"

https://github.com/jahstreet/spark-on-kubernetes-helm/blob/96f2521b0e20f30fe2dcede5c81239698bd01fb0/charts/jupyter-sparkmagic/templates/deployment.yaml#L2

Deprecated APIs Removed In 1.16: https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/

jahstreet commented 3 years ago

Thanks for reporting the issue. 1.0.0 charts are built for Spark v2.4.5- (which depends on fabric8 Java Kubernetes client 4.6.1 compatible with K8s API v1.15.3-). I'm preparing charts 2.0.0 for Spark 3.x, which are going to address such issues and extend the supported K8s API versions.

jahstreet commented 3 years ago

Going to be solved in #39 .

pdambrauskas commented 3 years ago

@jahstreet, have you decided not to support jupiter anymore? I see you released new versions for other charts, but jupiter-sparkmagic is missing.

jahstreet commented 3 years ago

@pdambrauskas , yeah, I've dropped it since spark-cluster contains the JupyterHub, in favour of which I would love to continue support. Also JupyterHub seems to be better alternative to standalone Jupyter on K8s.