actions / actions-runner-controller

Kubernetes controller for GitHub Actions self-hosted runners
Apache License 2.0
4.41k stars 1.04k forks source link

ARC with customizing the init container is throwing init/error #3457

Closed gpadmesh closed 2 months ago

gpadmesh commented 2 months ago

Checks

Controller Version

0.9.1

Deployment Method

Helm

Checks

To Reproduce

Here are the steps we followed.

1. Created a docker image runner-new with name and uploaded to our internal docker registry i.e. githubactions-docker.artifactory.org.com/runner-new:latest. See values.yaml file pointing to this docker image by composing from this link https://github.com/actions/runner/blob/main/images/Dockerfile and we added two additional packages on 11th line i.e RUN apt update -y && apt install curl unzip git zip -y
2. Helm install the controller set and runner scale set
3. Arc controllers set pod comes up clean but runner scale set pod throws an error message init/error
4. Attached pod description status

Deployment Commands

NAMESPACE="dev-arc-system"
helm install arc \
    --namespace "${NAMESPACE}" \
    --create-namespace \
    oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller

INSTALLATION_NAME="det-ubuntu-latest"
NAMESPACE="det-ubuntu-latest"
GITHUB_CONFIG_URL=https://github.com/XXXX
GITHUB_PAT="xxxxxxxxx"
helm install "${INSTALLATION_NAME}" \
    --namespace "${NAMESPACE}" \
    --create-namespace \
    --set githubConfigUrl="${GITHUB_CONFIG_URL}" \
    --set githubConfigSecret.github_token="${GITHUB_PAT}" \
    oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set -f  ~/Documents/arc-configs/values-det.yaml

Describe the bug

We have the challenge of customizing the init container. How do we solve it? Can we get good documentation? If you solve this problem, I will help create a step-by-step documentation.

Bug: Pod shows as init/error message

Describe the expected behavior

We want init container to with our own docker compose image built from https://github.com/actions/runner/blob/main/images/Dockerfile with few additional packages on 11th line i.e RUN apt update -y && apt install curl unzip git zip -y

Additional Context

## githubConfigUrl is the GitHub url for where you want to configure runners
## ex: https://github.com/myorg/myrepo or https://github.com/myorg
githubConfigUrl: "https://github.com/my-org"

## githubConfigSecret is the k8s secrets to use when auth with GitHub API.
## You can choose to use GitHub App or a PAT token
githubConfigSecret: 
  ### GitHub Apps Configuration
  ## NOTE: IDs MUST be strings, use quotes
  #github_app_id: ""
  #github_app_installation_id: ""
  #github_app_private_key: |

  ### GitHub PAT Configuration
  github_token: ""
## If you have a pre-define Kubernetes secret in the same namespace the gha-runner-scale-set is going to deploy,
## you can also reference it via `githubConfigSecret: pre-defined-secret`.
## You need to make sure your predefined secret has all the required secret data set properly.
##   For a pre-defined secret using GitHub PAT, the secret needs to be created like this:
##   > kubectl create secret generic pre-defined-secret --namespace=my_namespace --from-literal=github_token='ghp_your_pat'
##   For a pre-defined secret using GitHub App, the secret needs to be created like this:
##   > kubectl create secret generic pre-defined-secret --namespace=my_namespace --from-literal=github_app_id=123456 --from-literal=github_app_installation_id=654321 --from-literal=github_app_private_key='-----BEGIN CERTIFICATE-----*******'
# githubConfigSecret: pre-defined-secret

## proxy can be used to define proxy settings that will be used by the
## controller, the listener and the runner of this scale set.
#
# proxy:
#   http:
#     url: http://proxy.com:1234
#     credentialSecretRef: proxy-auth # a secret with `username` and `password` keys
#   https:
#     url: http://proxy.com:1234
#     credentialSecretRef: proxy-auth # a secret with `username` and `password` keys
#   noProxy:
#     - example.com
#     - example.org

## maxRunners is the max number of runners the autoscaling runner set will scale up to.
#maxRunners: 5

## minRunners is the min number of idle runners. The target number of runners created will be
## calculated as a sum of minRunners and the number of jobs assigned to the scale set.
minRunners: 2

# runnerGroup: "default"
runnerGroup: "det-ubuntu-latest"

## name of the runner scale set to create.  Defaults to the helm release name
# runnerScaleSetName: ""

## A self-signed CA certificate for communication with the GitHub server can be
## provided using a config map key selector. If `runnerMountPath` is set, for
## each runner pod ARC will:
## - create a `github-server-tls-cert` volume containing the certificate
##   specified in `certificateFrom`
## - mount that volume on path `runnerMountPath`/{certificate name}
## - set NODE_EXTRA_CA_CERTS environment variable to that same path
## - set RUNNER_UPDATE_CA_CERTS environment variable to "1" (as of version
##   2.303.0 this will instruct the runner to reload certificates on the host)
##
## If any of the above had already been set by the user in the runner pod
## template, ARC will observe those and not overwrite them.
## Example configuration:
#
# githubServerTLS:
#   certificateFrom:
#     configMapKeyRef:
#       name: config-map-name
#       key: ca.crt
#   runnerMountPath: /usr/local/share/ca-certificates/

## Container mode is an object that provides out-of-box configuration
## for dind and kubernetes mode. Template will be modified as documented under the
## template object.
##
## If any customization is required for dind or kubernetes mode, containerMode should remain
## empty, and configuration should be applied to the template.
containerMode:
  type: "dind"
# containerMode:
#   type: "dind"  ## type can be set to dind or kubernetes
#   ## the following is required when containerMode.type=kubernetes
#   kubernetesModeWorkVolumeClaim:
#     accessModes: ["ReadWriteOnce"]
#     # For local testing, use https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/quickstart.md to provide dynamic provision volume with storageClassName: openebs-hostpath
#     storageClassName: "dynamic-blob-storage"
#     resources:
#       requests:
#         storage: 1Gi
#   kubernetesModeServiceAccount:
#     annotations:

## template is the PodSpec for each listener Pod
## For reference: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec
# listenerTemplate:
#   spec:
#     containers:
#     # Use this section to append additional configuration to the listener container.
#     # If you change the name of the container, the configuration will not be applied to the listener,
#     # and it will be treated as a side-car container.
#     - name: listener
#       securityContext:
#         runAsUser: 1000
#     # Use this section to add the configuration of a side-car container.
#     # Comment it out or remove it if you don't need it.
#     # Spec for this container will be applied as is without any modifications.
#     - name: side-car
#       image: example-sidecar

## template is the PodSpec for each runner Pod
## For reference: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec
template:
  ## template.spec will be modified if you change the container mode
  ## with containerMode.type=dind, we will populate the template.spec with following pod spec
  ######################################################################################################
  ## with containerMode.type=kubernetes, we will populate the template.spec with following pod spec
  ## template:
  ##   spec:
  ##     containers:
  ##     - name: runner
  ##       image: ghcr.io/actions/actions-runner:latest
  ##       command: ["/home/runner/run.sh"]
  ##       env:
  ##         - name: ACTIONS_RUNNER_CONTAINER_HOOKS
  ##           value: /home/runner/k8s/index.js
  ##         - name: ACTIONS_RUNNER_POD_NAME
  ##           valueFrom:
  ##             fieldRef:
  ##               fieldPath: metadata.name
  ##         - name: ACTIONS_RUNNER_REQUIRE_JOB_CONTAINER
  ##           value: "true"
  ##       volumeMounts:
  ##         - name: work
  ##           mountPath: /home/runner/_work
  ##     volumes:
  ##       - name: work
  ##         ephemeral:
  ##           volumeClaimTemplate:
  ##             spec:
  ##               accessModes: [ "ReadWriteOnce" ]
  ##               storageClassName: "local-path"
  ##               resources:
  ##                 requests:
  ##                   storage: 1Gi
   spec:
     containers:
       - name: runner
         image: githubactions-docker.artifactory.org.com/runner-new:latest
         command: ["/home/runner/run.sh"]
##         resources:
##          limits:
##            cpu: 500m
##            memory: 512Mi

## Optional controller service account that needs to have required Role and RoleBinding
## to operate this gha-runner-scale-set installation.
## The helm chart will try to find the controller deployment and its service account at installation time.
## In case the helm chart can't find the right service account, you can explicitly pass in the following value
## to help it finish RoleBinding with the right service account.
## Note: if your controller is installed to only watch a single namespace, you have to pass these values explicitly.
 #    controllerServiceAccount:
 #      namespace: det-arc
 #      name: det-arc-system-gha-rs-controller

Controller Logs

2024-04-17T17:59:56Z    INFO    AutoscalingRunnerSet    Adding finalizer    {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:56Z    INFO    AutoscalingRunnerSet    Successfully added finalizer    {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:56Z    INFO    AutoscalingRunnerSet    Runner scale set id annotation does not exist. Creating a new runner scale set. {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:56Z    INFO    AutoscalingRunnerSet    Creating a new runner scale set {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:56Z    INFO    actions-clients retrieve actions client {"githubConfigURL": "https://github.com/org", "namespace": "det-ubuntu-latest"}
2024-04-17T17:59:56Z    INFO    actions-clients creating new client {"githubConfigURL": "https://github.com/org", "namespace": "det-ubuntu-latest"}
2024-04-17T17:59:56Z    INFO    actions-clients successfully created new client {"githubConfigURL": "https://github.com/org", "namespace": "det-ubuntu-latest"}
2024-04-17T17:59:56Z    INFO    actions-clients refreshing token    {"githubConfigUrl": "https://github.com/org"}
2024-04-17T17:59:56Z    INFO    actions-clients getting runner registration token   {"registrationTokenURL": "https://api.github.com/orgs/org/actions/runners/registration-token"}
2024-04-17T17:59:56Z    INFO    actions-clients getting Actions tenant URL and JWT  {"registrationURL": "https://api.github.com/actions/runner-registration"}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Created/Reused a runner scale set   {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "id": 9, "runnerGroupName": "det-ubuntu-latest"}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Adding runner scale set ID, name and runner group name as an annotation and url labels  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Updated with runner scale set ID, name and runner group name as an annotation   {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "id": 9, "name": "det-ubuntu-latest", "runnerGroupName": "det-ubuntu-latest"}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Latest runner set does not exist. Creating a new runner set.    {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Creating a new EphemeralRunnerSet resource  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Created a new EphemeralRunnerSet resource   {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t"}
2024-04-17T17:59:57Z    INFO    EphemeralRunnerSet  Adding finalizer    {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Find existing ephemeral runner set  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t", "specHash": "74f74cc7bc"}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    AutoscalingListener does not exist. {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Creating a new AutoscalingListener for the runner set   {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "ephemeralRunnerSetName": "det-ubuntu-latest-sf84t"}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Creating a new AutoscalingListener resource {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-6f7586ff-listener", "namespace": "dev-arc-system"}
2024-04-17T17:59:57Z    INFO    EphemeralRunnerSet  Successfully added finalizer    {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}}
2024-04-17T17:59:57Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 0, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T17:59:57Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 0, "desired": 0}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Created a new AutoscalingListener resource  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-6f7586ff-listener", "namespace": "dev-arc-system"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Adding finalizer    {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Find existing ephemeral runner set  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t", "specHash": "74f74cc7bc"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Successfully added finalizer    {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Find existing ephemeral runner set  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t", "specHash": "74f74cc7bc"}
2024-04-17T17:59:57Z    INFO    AutoscalingRunnerSet    Find existing ephemeral runner set  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t", "specHash": "74f74cc7bc"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating a mirror listener secret for the listener pod  {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating listener secret    {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Created listener secret {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Updating mirror listener secret for the listener pod    {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "mirrorSecretDataHash": "6b66bf5d66", "secretDataHash": "78bdc75994"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Updating listener mirror secret {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener", "hash": "78bdc75994"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Updated listener mirror secret  {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener", "hash": "78bdc75994"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating a service account for the listener pod {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating listener service accounts  {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Created listener service accounts   {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating a role for the listener pod    {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating listener role  {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "det-ubuntu-latest", "name": "det-ubuntu-latest-6f7586ff-listener", "rules": [{"verbs":["patch"],"apiGroups":["actions.github.com"],"resources":["ephemeralrunnersets"],"resourceNames":["det-ubuntu-latest-sf84t"]},{"verbs":["patch"],"apiGroups":["actions.github.com"],"resources":["ephemeralrunners","ephemeralrunners/status"]}]}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Created listener role   {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "det-ubuntu-latest", "name": "det-ubuntu-latest-6f7586ff-listener", "rules": [{"verbs":["patch"],"apiGroups":["actions.github.com"],"resources":["ephemeralrunnersets"],"resourceNames":["det-ubuntu-latest-sf84t"]},{"verbs":["patch"],"apiGroups":["actions.github.com"],"resources":["ephemeralrunners","ephemeralrunners/status"]}]}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating a role binding for the service account and role    {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating listener role binding  {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "det-ubuntu-latest", "name": "det-ubuntu-latest-6f7586ff-listener", "role": "det-ubuntu-latest-6f7586ff-listener", "serviceAccountNamespace": "dev-arc-system", "serviceAccount": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Created listener role binding   {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "det-ubuntu-latest", "name": "det-ubuntu-latest-6f7586ff-listener", "role": "det-ubuntu-latest-6f7586ff-listener", "serviceAccountNamespace": "dev-arc-system", "serviceAccount": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating a listener pod {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating listener config secret {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating a listener pod {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Creating listener pod   {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T17:59:57Z    INFO    AutoscalingListener Created listener pod    {"autoscalinglistener": {"name":"det-ubuntu-latest-6f7586ff-listener","namespace":"dev-arc-system"}, "namespace": "dev-arc-system", "name": "det-ubuntu-latest-6f7586ff-listener"}
2024-04-17T18:00:02Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 0, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:02Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 0, "desired": 2}
2024-04-17T18:00:02Z    INFO    EphemeralRunnerSet  Creating new ephemeral runners (scale up)   {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "count": 2}
2024-04-17T18:00:02Z    INFO    EphemeralRunnerSet  Creating new ephemeral runner   {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "progress": 1, "total": 2}
2024-04-17T18:00:02Z    INFO    AutoscalingRunnerSet    Find existing ephemeral runner set  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t", "specHash": "74f74cc7bc"}
2024-04-17T18:00:02Z    INFO    EphemeralRunnerSet  Created new ephemeral runner    {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "runner": "det-ubuntu-latest-sf84t-runner-948v7"}
2024-04-17T18:00:02Z    INFO    EphemeralRunnerSet  Creating new ephemeral runner   {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "progress": 2, "total": 2}
2024-04-17T18:00:02Z    INFO    EphemeralRunner Adding runner registration finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Created new ephemeral runner    {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "runner": "det-ubuntu-latest-sf84t-runner-2kph9"}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Updating status with current runners count  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "count": 2}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Successfully added runner registration finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Adding finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Successfully added finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Adding runner registration finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    AutoscalingRunnerSet    Find existing ephemeral runner set  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t", "specHash": "74f74cc7bc"}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Successfully added runner registration finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Adding finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    AutoscalingRunnerSet    Find existing ephemeral runner set  {"autoscalingrunnerset": {"name":"det-ubuntu-latest","namespace":"det-ubuntu-latest"}, "name": "det-ubuntu-latest-sf84t", "specHash": "74f74cc7bc"}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Successfully added finalizer    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new ephemeral runner registration and updating status with runner config   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating ephemeral runner JIT config    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    actions-clients retrieve actions client {"githubConfigURL": "https://github.com/org", "namespace": "det-ubuntu-latest"}
2024-04-17T18:00:03Z    INFO    actions-clients using cache client  {"githubConfigURL": "https://github.com/org", "namespace": "det-ubuntu-latest"}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created ephemeral runner JIT config {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}, "runnerId": 489}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updating ephemeral runner status with runnerId and runnerJITConfig  {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updated ephemeral runner status with runnerId and runnerJITConfig   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new ephemeral runner registration and updating status with runner config   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating ephemeral runner JIT config    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    actions-clients retrieve actions client {"githubConfigURL": "https://github.com/org", "namespace": "det-ubuntu-latest"}
2024-04-17T18:00:03Z    INFO    actions-clients using cache client  {"githubConfigURL": "https://github.com/org", "namespace": "det-ubuntu-latest"}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created ephemeral runner JIT config {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}, "runnerId": 490}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updating ephemeral runner status with runnerId and runnerJITConfig  {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updated ephemeral runner status with runnerId and runnerJITConfig   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new ephemeral runner secret for jitconfig. {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new secret for ephemeral runner    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created new secret spec for ephemeral runner    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created ephemeral runner secret {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}, "secretName": "det-ubuntu-latest-sf84t-runner-948v7"}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new ephemeral runner secret for jitconfig. {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new secret for ephemeral runner    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created new secret spec for ephemeral runner    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created ephemeral runner secret {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}, "secretName": "det-ubuntu-latest-sf84t-runner-2kph9"}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new EphemeralRunner pod.   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new pod for ephemeral runner   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created new pod spec for ephemeral runner   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created ephemeral runner pod    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}, "runnerScaleSetId": 9, "runnerName": "det-ubuntu-latest-sf84t-runner-948v7", "runnerId": 489, "configUrl": "https://github.com/org", "podName": "det-ubuntu-latest-sf84t-runner-948v7"}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new EphemeralRunner pod.   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Creating new pod for ephemeral runner   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created new pod spec for ephemeral runner   {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Created ephemeral runner pod    {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}, "runnerScaleSetId": 9, "runnerName": "det-ubuntu-latest-sf84t-runner-2kph9", "runnerId": 490, "configUrl": "https://github.com/org", "podName": "det-ubuntu-latest-sf84t-runner-2kph9"}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Waiting for runner container status to be available {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Waiting for runner container status to be available {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Waiting for runner container status to be available {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updating ephemeral runner status with pod phase {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}, "statusPhase": "Pending", "statusReason": "", "statusMessage": ""}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updated ephemeral runner status with pod phase  {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updating ephemeral runner status with pod phase {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}, "statusPhase": "Pending", "statusReason": "", "statusMessage": ""}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Updated ephemeral runner status with pod phase  {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Ephemeral runner counts {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "pending": 2, "running": 0, "finished": 0, "failed": 0, "deleting": 0}
2024-04-17T18:00:03Z    INFO    EphemeralRunnerSet  Scaling comparison  {"ephemeralrunnerset": {"name":"det-ubuntu-latest-sf84t","namespace":"det-ubuntu-latest"}, "current": 2, "desired": 2}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:03Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}

2024-04-17T18:00:38Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:39Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:41Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}
2024-04-17T18:00:41Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-2kph9","namespace":"det-ubuntu-latest"}}

2024-04-17T18:01:07Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:01:09Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}
2024-04-17T18:01:09Z    INFO    EphemeralRunner Ephemeral runner container is still running {"ephemeralrunner": {"name":"det-ubuntu-latest-sf84t-runner-948v7","namespace":"det-ubuntu-latest"}}

Runner Pod Logs

Defaulted container "runner" out of: runner, dind, init-dind-externals (init)
Error from server (BadRequest): container "runner" in pod "det-ubuntu-latest-sf84t-runner-2kph9" is waiting to start: PodInitializing
gpadmesh commented 2 months ago
image
gpadmesh commented 2 months ago

Pod description

Defaulted container "runner" out of: runner, dind, init-dind-externals (init) Error from server (BadRequest): container "runner" in pod "det-ubuntu-latest-sf84t-runner-2kph9" is waiting to start: PodInitializing

% k describe pod det-ubuntu-latest-sf84t-runner-2kph9 -n det-ubuntu-latest Name: det-ubuntu-latest-sf84t-runner-2kph9 Namespace: det-ubuntu-latest Priority: 0 Service Account: det-ubuntu-latest-gha-rs-no-permission Node: gha-dev-vs-cc-cluster-md-0-tfkjk-f8gbk/10.124.66.101 Start Time: Wed, 17 Apr 2024 14:00:03 -0400 Labels: actions-ephemeral-runner=True actions.github.com/organization=es-det actions.github.com/scale-set-name=det-ubuntu-latest actions.github.com/scale-set-namespace=det-ubuntu-latest app.kubernetes.io/component=runner app.kubernetes.io/part-of=gha-runner-scale-set app.kubernetes.io/version=0.9.1 pod-template-hash=ff8579457 Annotations: actions.github.com/patch-id: 0 actions.github.com/runner-group-name: det-ubuntu-latest actions.github.com/runner-scale-set-name: det-ubuntu-latest actions.github.com/runner-spec-hash: 74f74cc7bc Status: Failed IP: 10.0.0.0 IPs: IP: 10.0.0.0 Controlled By: EphemeralRunner/det-ubuntu-latest-sf84t-runner-2kph9 Init Containers: init-dind-externals: Container ID: containerd://95af60d993affc18d118670b66519f6c280fa5747797690bdf4b878c7112c401 Image: githubactions-docker.artifactory.org.com/org-runner-new:latest Image ID: githubactions-docker.artifactory.org.com/org-runner-new@sha256:1dd522346b21b80ac1967fd8fb74cb89acdd70498ed2273e9198dd2e3227a020 Port: Host Port: Command: cp Args: -r -v /home/runner/externals/. /home/runner/tmpDir/ State: Terminated Reason: Error Exit Code: 1 Started: Wed, 17 Apr 2024 14:00:38 -0400 Finished: Wed, 17 Apr 2024 14:00:38 -0400 Ready: False Restart Count: 0 Environment: Mounts: /home/runner/tmpDir from dind-externals (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vcxkv (ro) Containers: runner: Container ID:
Image: githubactions-docker.artifactory.org.com/org-runner-new:latest Image ID:
Port: Host Port: Command: /home/runner/run.sh State: Waiting Reason: PodInitializing Ready: False Restart Count: 0 Environment: DOCKER_HOST: unix:///var/run/docker.sock RUNNER_WAIT_FOR_DOCKER_IN_SECONDS: 120 ACTIONS_RUNNER_INPUT_JITCONFIG: <set to the key 'jitToken' in secret 'det-ubuntu-latest-sf84t-runner-2kph9'> Optional: false GITHUB_ACTIONS_RUNNER_EXTRA_USER_AGENT: actions-runner-controller/0.9.1 Mounts: /home/runner/_work from work (rw) /var/run from dind-sock (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vcxkv (ro) dind: Container ID:
Image: docker:dind Image ID:
Port: Host Port: Args: dockerd --host=unix:///var/run/docker.sock --group=$(DOCKER_GROUP_GID) State: Waiting Reason: PodInitializing Ready: False Restart Count: 0 Environment: DOCKER_GROUP_GID: 123 Mounts: /home/runner/_work from work (rw) /home/runner/externals from dind-externals (rw) /var/run from dind-sock (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vcxkv (ro) Conditions: Type Status Initialized False Ready False ContainersReady False PodScheduled True Volumes: dind-sock: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium:
SizeLimit: dind-externals: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium:
SizeLimit: work: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium:
SizeLimit: kube-api-access-vcxkv: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events:

nikola-jokic commented 2 months ago

Hey @gpadmesh,

We documented how to customize container modes here. Since this issue is a question, I will close it now. Please use discussions for questions.