cant create POD in namepsace - Caused by: java.net.SocketTimeoutException: timeout

[sreejesh-radhakrishnan-db]

Getting below error:

Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Operation: [create] for kind: [Pod] with name: [null] in namespace: [spark-operator] failed. at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:349) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:84) at org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:139) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$3(KubernetesClientApplication.scala:213) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$3$adapted(KubernetesClientApplication.scala:207) at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2611) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:207) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:179) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.net.SocketTimeoutException: timeout

I am using webhook enables and its a private GKE hence have passed 443 as well: helm upgrade --install gpds spark-operator/spark-operator --namespace spark-operator \ --set sparkJobNamespace=spark-operator --set webhook.enable=true --set webhook.port=443 \ -f ./utils/spark-operator/values.yaml

Deployment setting:

spec: containers:

• args:
  • -v=2
  • -logtostderr
  • -namespace=spark-operator
  • -enable-ui-service=true
  • -ingress-url-format=
  • -controller-threads=10
  • -resync-interval=30
  • -enable-batch-scheduler=true
  • -label-selector-filter=
  • -enable-metrics=true
  • -metrics-labels=app_type
  • -metrics-port=10254
  • -metrics-endpoint=/metrics
  • -metrics-prefix=
  • -enable-webhook=true
  • -webhook-svc-namespace=spark-operator
  • -webhook-port=443
  • -webhook-timeout=30
  • -webhook-svc-name=gpds-spark-operator-webhook
  • -webhook-config-name=gpds-spark-operator-webhook-config
  • -webhook-namespace-selector=
  • -enable-resource-quota-enforcement=false any help please?

[sreejesh-radhakrishnan-db]

I did look at similar error- but I dount this is the root cause or, https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/1478

also not great info as to how this was fixed

[sreejesh-radhakrishnan-db]

can anyone help please? compeltly stuck.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.