SecurityContext don't pass into the executor pods

[lybavsky]

Hello everyone. We've found out that securitycontext which we put into sparkapplication custom resource like that: securityContext: allowPrivilegeEscalation: true runAsUser: 1000 capabilities: add: [ "SYS_PTRACE" ]

does not present on created executor pods: securityContext: fsGroup: 1 runAsUser: 1000 seccompProfile: type: RuntimeDefault supplementalGroups:

• 1

We are using 3.1.1 spark version and sparkoperator (spark-operator-1.1.6, v1beta2-1.2.3-3.1.1)

On master branch in this repo we see that addSecurityContext in patch.go use default container name only "executor", while 3rd version of spark using "spark-kubernetes-executor" name. It is possible to use function "findContainer", but it not using now.

Please review my merge request. https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/pull/1377

What the way to solve our problem? Thank you

[seanturner026]

Having the same issue. The mutation is running, and the driver pod is running. However, the mutation doesn't appear to be able to find the execution pod and add allowPrivilegeEscalation = false. This causes OPA to block the executor from launching as allowPrivilegeEscalation isn't in the executor pod.

$ kubectl logs spark-operator-6f8f7b8c98-qgjk7

``` I1104 19:28:30.939013 10 webhook.go:556] Pod pythonpi-6a3b347cec60b782-exec-737 in namespace spark is subject to mutation I1104 19:28:31.971699 10 webhook.go:246] Serving admission request W1104 19:28:31.972332 10 patch.go:619] Spark driver/executor container not found in pod pythonpi-6a3b347cec60b782-exec-738 I1104 19:28:31.972458 10 webhook.go:556] Pod pythonpi-6a3b347cec60b782-exec-738 in namespace spark is subject to mutation I1104 19:28:33.016165 10 webhook.go:246] Serving admission request W1104 19:28:33.016755 10 patch.go:619] Spark driver/executor container not found in pod pythonpi-6a3b347cec60b782-exec-739 I1104 19:28:33.016825 10 webhook.go:556] Pod pythonpi-6a3b347cec60b782-exec-739 in namespace spark is subject to mutation I1104 19:28:34.092768 10 webhook.go:246] Serving admission request W1104 19:28:34.093520 10 patch.go:619] Spark driver/executor container not found in pod pythonpi-6a3b347cec60b782-exec-740 I1104 19:28:34.093535 10 webhook.go:556] Pod pythonpi-6a3b347cec60b782-exec-740 in namespace spark is subject to mutation I1104 19:28:35.132686 10 webhook.go:246] Serving admission request W1104 19:28:35.133213 10 patch.go:619] Spark driver/executor container not found in pod pythonpi-6a3b347cec60b782-exec-741 I1104 19:28:35.133231 10 webhook.go:556] Pod pythonpi-6a3b347cec60b782-exec-741 in namespace spark is subject to mutation I1104 19:28:36.165891 10 webhook.go:246] Serving admission request W1104 19:28:36.166388 10 patch.go:619] Spark driver/executor container not found in pod pythonpi-6a3b347cec60b782-exec-742 I1104 19:28:36.166407 10 webhook.go:556] Pod pythonpi-6a3b347cec60b782-exec-742 in namespace spark is subject to mutation I1104 19:28:37.207682 10 webhook.go:246] Serving admission request ```

$ k logs pyspark-pi-driver

``` ++ id -u + myuid=185 ++ id -g + mygid=0 + set +e ++ getent passwd 185 + uidentry= + set -e + '[' -z '' ']' + '[' -w /etc/passwd ']' + echo '185:x:185:0:anonymous uid:/opt/spark:/bin/false' + SPARK_CLASSPATH=':/opt/spark/jars/*' + env + sed 's/[^=]*=\(.*\)/\1/g' + sort -t_ -k4 -n + grep SPARK_JAVA_OPT_ + readarray -t SPARK_EXECUTOR_JAVA_OPTS + '[' -n '' ']' + '[' -z ']' + '[' -z ']' + '[' -n '' ']' + '[' -z ']' + '[' -z x ']' + SPARK_CLASSPATH='/opt/spark/conf::/opt/spark/jars/*' + case "$1" in + shift 1 + CMD=("$SPARK_HOME/bin/spark-submit" --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" --deploy-mode client "$@") + exec /usr/bin/tini -s -- /opt/spark/bin/spark-submit --conf spark.driver.bindAddress=10.83.4.110 --deploy-mode client --properties-file /opt/spark/conf/spark.properties --class org.apache.spark.deploy.PythonRunner local:///opt/spark/examples/src/main/python/pi.py WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark/jars/spark-unsafe_2.12-3.1.1.jar) to constructor java.nio.DirectByteBuffer(long,int) WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 21/11/04 19:38:28 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 21/11/04 19:38:29 INFO SparkContext: Running Spark version 3.1.1 21/11/04 19:38:29 INFO ResourceUtils: ============================================================== 21/11/04 19:38:29 INFO ResourceUtils: No custom resources configured for spark.driver. 21/11/04 19:38:29 INFO ResourceUtils: ============================================================== 21/11/04 19:38:29 INFO SparkContext: Submitted application: PythonPi 21/11/04 19:38:29 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 512, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0) 21/11/04 19:38:29 INFO ResourceProfile: Limiting resource is cpus at 1 tasks per executor 21/11/04 19:38:29 INFO ResourceProfileManager: Added ResourceProfile id: 0 21/11/04 19:38:29 INFO SecurityManager: Changing view acls to: 185 21/11/04 19:38:29 INFO SecurityManager: Changing modify acls to: 185 21/11/04 19:38:29 INFO SecurityManager: Changing view acls groups to: 21/11/04 19:38:29 INFO SecurityManager: Changing modify acls groups to: 21/11/04 19:38:29 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(185); groups with view permissions: Set(); users with modify permissions: Set(185); groups with modify permissions: Set() 21/11/04 19:38:30 INFO Utils: Successfully started service 'sparkDriver' on port 7078. 21/11/04 19:38:30 INFO SparkEnv: Registering MapOutputTracker 21/11/04 19:38:30 INFO SparkEnv: Registering BlockManagerMaster 21/11/04 19:38:30 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information 21/11/04 19:38:30 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up 21/11/04 19:38:30 INFO SparkEnv: Registering BlockManagerMasterHeartbeat 21/11/04 19:38:30 INFO DiskBlockManager: Created local directory at /var/data/spark-07fd5498-50b6-48e0-9bf8-5c2930c724d6/blockmgr-aa33d8de-3c46-4e36-a8f8-a09a721a36af 21/11/04 19:38:30 INFO MemoryStore: MemoryStore started with capacity 117.0 MiB 21/11/04 19:38:30 INFO SparkEnv: Registering OutputCommitCoordinator 21/11/04 19:38:31 INFO Utils: Successfully started service 'SparkUI' on port 4040. 21/11/04 19:38:31 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://pyspark-pi-d952757cec759368-driver-svc.spark.svc:4040 21/11/04 19:38:31 INFO SparkKubernetesClientFactory: Auto-configuring K8S client using current context from users K8S config file 21/11/04 19:38:34 INFO ExecutorPodsAllocator: Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 1 running: 0. 21/11/04 19:38:34 INFO BasicExecutorFeatureStep: Decommissioning not enabled, skipping shutdown script 21/11/04 19:38:34 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 7079. 21/11/04 19:38:34 INFO NettyBlockTransferService: Server created on pyspark-pi-d952757cec759368-driver-svc.spark.svc:7079 21/11/04 19:38:34 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 21/11/04 19:38:34 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, pyspark-pi-d952757cec759368-driver-svc.spark.svc, 7079, None) 21/11/04 19:38:34 INFO BlockManagerMasterEndpoint: Registering block manager pyspark-pi-d952757cec759368-driver-svc.spark.svc:7079 with 117.0 MiB RAM, BlockManagerId(driver, pyspark-pi-d952757cec759368-driver-svc.spark.svc, 7079, None) 21/11/04 19:38:34 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, pyspark-pi-d952757cec759368-driver-svc.spark.svc, 7079, None) 21/11/04 19:38:34 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, pyspark-pi-d952757cec759368-driver-svc.spark.svc, 7079, None) 21/11/04 19:38:34 WARN ExecutorPodsSnapshotsStoreImpl: Exception when notifying snapshot subscriber. io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://kubernetes.default.svc/api/v1/namespaces/spark/pods. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. admission webhook "validation.gatekeeper.sh" denied the request: [privilege-escalation] Privilege escalation container is not allowed: spark-kubernetes-executor. at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:589) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:526) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:492) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:451) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:252) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:879) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:341) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:84) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$requestNewExecutors$2(ExecutorPodsAllocator.scala:307) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.requestNewExecutors(ExecutorPodsAllocator.scala:291) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$onNewSnapshots$10(ExecutorPodsAllocator.scala:256) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$onNewSnapshots$10$adapted(ExecutorPodsAllocator.scala:185) at scala.collection.Iterator.foreach(Iterator.scala:941) at scala.collection.Iterator.foreach$(Iterator.scala:941) at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.onNewSnapshots(ExecutorPodsAllocator.scala:185) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$start$1(ExecutorPodsAllocator.scala:98) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$start$1$adapted(ExecutorPodsAllocator.scala:98) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsSnapshotsStoreImpl$SnapshotsSubscriber.org$apache$spark$scheduler$cluster$k8s$ExecutorPodsSnapshotsStoreImpl$SnapshotsSubscriber$$processSnapshotsInternal(ExecutorPodsSnapshotsStoreImpl.scala:138) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsSnapshotsStoreImpl$SnapshotsSubscriber.processSnapshots(ExecutorPodsSnapshotsStoreImpl.scala:126) at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsSnapshotsStoreImpl.$anonfun$addSubscriber$1(ExecutorPodsSnapshotsStoreImpl.scala:81) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) ```

The error in there is Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. admission webhook "validation.gatekeeper.sh" denied the request: [privilege-escalation] Privilege escalation container is not allowed: spark-kubernetes-executor. This is saying that the spark-kubernetes-executor is trying to run with privilege escalation.

spark-py-pi.yaml

```yaml # # Copyright 2018 Google LLC # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # Support for Python is experimental, and requires building SNAPSHOT image of Apache Spark, # with `imagePullPolicy` set to Always apiVersion: "sparkoperator.k8s.io/v1beta2" kind: SparkApplication metadata: name: pyspark-pi namespace: spark spec: type: Python pythonVersion: "3" mode: cluster image: "gcr.io/spark-operator/spark-py:v3.1.1" imagePullPolicy: Always mainApplicationFile: local:///opt/spark/examples/src/main/python/pi.py sparkVersion: "3.1.1" restartPolicy: type: OnFailure onFailureRetries: 3 onFailureRetryInterval: 10 onSubmissionFailureRetries: 5 onSubmissionFailureRetryInterval: 20 driver: cores: 1 coreLimit: "1200m" memory: "512m" labels: version: 3.1.1 serviceAccount: spark-operator-spark podSecurityContext: runAsUser: 185 securityContext: allowPrivilegeEscalation: false runAsNonRoot: true runAsUser: 185 executor: cores: 1 instances: 1 memory: "512m" labels: version: 3.1.1 podSecurityContext: runAsUser: 185 securityContext: allowPrivilegeEscalation: false runAsNonRoot: true runAsUser: 185 ```

helm values

```yaml istio: enabled: false securityContext: runAsNonRoot: true runAsUser: 185 allowPrivilegeEscalation: false webhook: enable: true ```

Running in spark namespace.

[seanturner026]

The executor pod is definitely not mounting the security context correctly. I removed OPA in the namespace and got an executor pod to launch:

$ k get pod pythonpi-08c5367cecc4dbf7-exec-1 -o yaml

```yaml kgp pythonpi-08c5367cecc4dbf7-exec-1 -o yaml ⎈ services-dev/spark apiVersion: v1 kind: Pod metadata: annotations: kubernetes.io/psp: eks.privileged creationTimestamp: "2021-11-04T21:05:00Z" labels: spark-app-selector: spark-432c09f701f5499bb291b9fd5f363705 spark-exec-id: "1" spark-exec-resourceprofile-id: "0" spark-role: executor sparkoperator.k8s.io/app-name: pyspark-pi sparkoperator.k8s.io/launched-by-spark-operator: "true" sparkoperator.k8s.io/submission-id: 38c7b085-6ce2-4959-b43a-0662528afd69 version: 3.1.1 managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:labels: .: {} f:spark-app-selector: {} f:spark-exec-id: {} f:spark-exec-resourceprofile-id: {} f:spark-role: {} f:sparkoperator.k8s.io/app-name: {} f:sparkoperator.k8s.io/launched-by-spark-operator: {} f:sparkoperator.k8s.io/submission-id: {} f:version: {} f:ownerReferences: .: {} k:{"uid":"fe180be4-1dd6-4811-88d9-6b5d6ad90d32"}: .: {} f:apiVersion: {} f:controller: {} f:kind: {} f:name: {} f:uid: {} f:spec: f:containers: k:{"name":"spark-kubernetes-executor"}: .: {} f:args: {} f:env: .: {} k:{"name":"SPARK_APPLICATION_ID"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_CONF_DIR"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_DRIVER_URL"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_EXECUTOR_CORES"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_EXECUTOR_ID"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_EXECUTOR_MEMORY"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_EXECUTOR_POD_IP"}: .: {} f:name: {} f:valueFrom: .: {} f:fieldRef: .: {} f:apiVersion: {} f:fieldPath: {} k:{"name":"SPARK_JAVA_OPT_0"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_JAVA_OPT_1"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_LOCAL_DIRS"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_RESOURCE_PROFILE_ID"}: .: {} f:name: {} f:value: {} k:{"name":"SPARK_USER"}: .: {} f:name: {} f:value: {} f:image: {} f:imagePullPolicy: {} f:name: {} f:ports: .: {} k:{"containerPort":7079,"protocol":"TCP"}: .: {} f:containerPort: {} f:name: {} f:protocol: {} f:resources: .: {} f:limits: .: {} f:memory: {} f:requests: .: {} f:cpu: {} f:memory: {} f:terminationMessagePath: {} f:terminationMessagePolicy: {} f:volumeMounts: .: {} k:{"mountPath":"/opt/spark/conf"}: .: {} f:mountPath: {} f:name: {} k:{"mountPath":"/var/data/spark-eb7caf1b-0310-48a3-842d-9ee2100a2e02"}: .: {} f:mountPath: {} f:name: {} f:dnsPolicy: {} f:enableServiceLinks: {} f:hostname: {} f:restartPolicy: {} f:schedulerName: {} f:securityContext: .: {} f:runAsUser: {} f:serviceAccount: {} f:serviceAccountName: {} f:terminationGracePeriodSeconds: {} f:volumes: .: {} k:{"name":"spark-conf-volume-exec"}: .: {} f:configMap: .: {} f:defaultMode: {} f:name: {} f:name: {} k:{"name":"spark-local-dir-1"}: .: {} f:emptyDir: {} f:name: {} manager: okhttp operation: Update time: "2021-11-04T21:04:59Z" - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:status: f:conditions: .: {} k:{"type":"PodScheduled"}: .: {} f:lastProbeTime: {} f:lastTransitionTime: {} f:message: {} f:reason: {} f:status: {} f:type: {} manager: kube-scheduler operation: Update time: "2021-11-04T21:05:00Z" - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:status: f:conditions: k:{"type":"ContainersReady"}: .: {} f:lastProbeTime: {} f:lastTransitionTime: {} f:message: {} f:reason: {} f:status: {} f:type: {} k:{"type":"Initialized"}: .: {} f:lastProbeTime: {} f:lastTransitionTime: {} f:status: {} f:type: {} k:{"type":"Ready"}: .: {} f:lastProbeTime: {} f:lastTransitionTime: {} f:message: {} f:reason: {} f:status: {} f:type: {} f:containerStatuses: {} f:hostIP: {} f:startTime: {} manager: kubelet operation: Update time: "2021-11-04T21:05:37Z" name: pythonpi-08c5367cecc4dbf7-exec-1 namespace: spark ownerReferences: - apiVersion: v1 controller: true kind: Pod name: pyspark-pi-driver uid: fe180be4-1dd6-4811-88d9-6b5d6ad90d32 resourceVersion: "59662865" uid: 0ec1e6f3-33c4-4916-a951-ddc0670480f2 spec: containers: - args: - executor env: - name: SPARK_USER value: "185" - name: SPARK_DRIVER_URL value: spark://CoarseGrainedScheduler@pyspark-pi-38957e7cecc4ae76-driver-svc.spark.svc:7078 - name: SPARK_EXECUTOR_CORES value: "1" - name: SPARK_EXECUTOR_MEMORY value: 512m - name: SPARK_APPLICATION_ID value: spark-432c09f701f5499bb291b9fd5f363705 - name: SPARK_CONF_DIR value: /opt/spark/conf - name: SPARK_EXECUTOR_ID value: "1" - name: SPARK_RESOURCE_PROFILE_ID value: "0" - name: SPARK_EXECUTOR_POD_IP valueFrom: fieldRef: apiVersion: v1 fieldPath: status.podIP - name: SPARK_JAVA_OPT_0 value: -Dspark.driver.blockManager.port=7079 - name: SPARK_JAVA_OPT_1 value: -Dspark.driver.port=7078 - name: SPARK_LOCAL_DIRS value: /var/data/spark-eb7caf1b-0310-48a3-842d-9ee2100a2e02 image: gcr.io/spark-operator/spark-py:v3.1.1 imagePullPolicy: Always name: spark-kubernetes-executor ports: - containerPort: 7079 name: blockmanager protocol: TCP resources: limits: memory: 896Mi requests: cpu: "1" memory: 896Mi terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /opt/spark/conf name: spark-conf-volume-exec - mountPath: /var/data/spark-eb7caf1b-0310-48a3-842d-9ee2100a2e02 name: spark-local-dir-1 - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-4rz76 readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true hostname: pythonpi-08c5367cecc4dbf7-exec-1 nodeName: ip-10-83-34-185.ec2.internal preemptionPolicy: PreemptLowerPriority priority: 0 restartPolicy: Never schedulerName: default-scheduler securityContext: runAsUser: 185 serviceAccount: spark-operator-spark serviceAccountName: spark-operator-spark terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes: - configMap: defaultMode: 420 name: spark-exec-32d8597cecc4e687-conf-map name: spark-conf-volume-exec - emptyDir: {} name: spark-local-dir-1 - name: kube-api-access-4rz76 projected: defaultMode: 420 sources: - serviceAccountToken: expirationSeconds: 3607 path: token - configMap: items: - key: ca.crt path: ca.crt name: kube-root-ca.crt - downwardAPI: items: - fieldRef: apiVersion: v1 fieldPath: metadata.namespace path: namespace status: conditions: - lastProbeTime: null lastTransitionTime: "2021-11-04T21:05:37Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2021-11-04T21:05:37Z" message: 'containers with unready status: [spark-kubernetes-executor]' reason: ContainersNotReady status: "False" type: Ready - lastProbeTime: null lastTransitionTime: "2021-11-04T21:05:37Z" message: 'containers with unready status: [spark-kubernetes-executor]' reason: ContainersNotReady status: "False" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2021-11-04T21:05:37Z" status: "True" type: PodScheduled containerStatuses: - image: gcr.io/spark-operator/spark-py:v3.1.1 imageID: "" lastState: {} name: spark-kubernetes-executor ready: false restartCount: 0 started: false state: waiting: reason: ContainerCreating hostIP: 10.83.34.185 phase: Pending qosClass: Burstable startTime: "2021-11-04T21:05:37Z" ```

From the above:

  securityContext:
    runAsUser: 185

[seanturner026]

https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/pull/1377 has been merged. I just removed the operator from my cluster and don't think I'll be able to test soon, but curious to see if the behavior changes.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.