search

keptn-contrib / job-executor-service

Running customizable tasks with Keptn as Kubernetes Jobs
Apache License 2.0
19 stars 12 forks source link

CreateContainerConfigError should exit immediately #254

Open agardnerIT opened 2 years ago

agardnerIT commented 2 years ago

Any pod where the init container fails will never be able to start. JES should fail quickly and not wait for the timeout.

Recreate Issue

Try to run a pod as root:

tasks:
      - name: "Do something"
        securityContext:
          runAsUser: 0
          runAsGroup: 0

Which results in:

Warning  Failed     13s (x12 over 2m27s)  kubelet            Error: container's runAsUser breaks non-root policy (pod: "job-executor-service-job-***(****)", container: init-job-executor-service-job-****1)
State:          Waiting
  Reason:       CreateContainerConfigError

Impact

Further tasks are blocked until the timeout (5mins by default).

christian-kreuzberger-dtx commented 2 years ago

Hi Adam,

I understand that this is an inconvenience right now. We can't fix this easily. We recently merged a PR https://github.com/keptn-contrib/job-executor-service/pull/249 that at least reports the correct error afterwards, but you will still run into the timeout.

The best way to fix this would be to refactor the Kubernetes Job Implementation (see https://github.com/keptn-contrib/job-executor-service/issues/244 ).

For now, what you could do is lower the timeout by setting maxPollDuration to something like 60 seconds: https://github.com/keptn-contrib/job-executor-service/blob/main/FEATURES.md#poll-duration