search

canonical / spark-operator

Spark Operator
Apache License 2.0
1 stars 2 forks source link

Spark Operator Charm Not Working with Kubernetes 1.22 #7

Closed stonepreston closed 2 years ago

stonepreston commented 2 years ago

When using Kubernetes 1.22, the spark operator does not deploy correctly:

$ charmcraft pack
$ juju deploy ./spark_ubuntu-20.04-amd64.charm --resource oci-image=gcr.io/spark-operator/spark-operator:v1beta2-1.2.3-3.1.1

The spark application has the following error state when using juju status:

App    Version                         Status  Scale  Charm  Store  Channel  Rev  OS          Address  Message
spark  .../spark-operator:v1beta2-...  error       1  spark  local             0  kubernetes           creating or updating custom resource definitions: ensuring custom resource definition "sparkapplications.sparkoperator.k8s.io" with version "v1beta1": the server could not find the requested resource

Unit      Workload  Agent  Address  Ports  Message
spark/0*  waiting   idle                   waiting for container

However, using kubernetes 1.21 the deploy succeeds without error:

App    Version                         Status  Scale  Charm  Store  Channel  Rev  OS          Address        Message
spark  .../spark-operator:v1beta2-...  active      1  spark  local             0  kubernetes  10.152.183.36  

Unit      Workload  Agent  Address      Ports              Message
spark/0*  active    idle   10.1.19.137  10254/TCP,443/TCP

and the crd that was missing in 1.22 is now present:

$ kubectl get crds | grep sparkapplication
scheduledsparkapplications.sparkoperator.k8s.io   2021-12-01T16:23:21Z
sparkapplications.sparkoperator.k8s.io            2021-12-01T16:23:20Z