openmetadata-dependencies pods are in pending state.

[ChiragGaur13]

Checks

• [X] I have checked for existing issues.
• [ ] This report is about the Openmetadata Helm Chart.
• [X] This report is about the Openmetadata Dependencies Helm Chart.

Description

Bug Description = I follow the installation doc of open metadata in Quickstart openmetadata in kubernetes. I follow every step one by one and it install openmetadata and its dependencies without any error but openmetadata pods are in pending state.

Steps to reproduce

No response

Chart Version

1.3.3

App Version

1.3.3

Kubernetes Version

v1.28.8

Helm Version

v3.9.3

Relevant Logs

kubectl get pods
NAME                                                      READY   STATUS     RESTARTS   AGE
my-permission-pod                                         1/1     Running    0          25h
mysql-0                                                   0/1     Pending    0          73s
nfs-server-ff44c5f84-5hbnv                                1/1     Running    0          25h
openmetadata-85b87d7b84-jdgsw                             0/1     Init:0/1   0          22s
openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7   0/1     Pending    0          73s
openmetadata-dependencies-scheduler-695fb566f7-kcl2g      0/1     Pending    0          73s
openmetadata-dependencies-sync-users-7596cd6bb6-w622x     0/1     Pending    0          73s
openmetadata-dependencies-triggerer-5dbc4cb67f-wgxfr      0/1     Pending    0          73s
openmetadata-dependencies-web-88b974456-8zkh6             0/1     Pending    0          73s
opensearch-0                                              0/1     Pending    0          73s
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ kubectl get all
NAME                                                          READY   STATUS     RESTARTS   AGE
pod/my-permission-pod                                         1/1     Running    0          25h
pod/mysql-0                                                   0/1     Pending    0          2m38s
pod/nfs-server-ff44c5f84-5hbnv                                1/1     Running    0          25h
pod/openmetadata-85b87d7b84-jdgsw                             0/1     Init:0/1   0          107s
pod/openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7   0/1     Pending    0          2m38s
pod/openmetadata-dependencies-scheduler-695fb566f7-kcl2g      0/1     Pending    0          2m38s
pod/openmetadata-dependencies-sync-users-7596cd6bb6-w622x     0/1     Pending    0          2m38s
pod/openmetadata-dependencies-triggerer-5dbc4cb67f-wgxfr      0/1     Pending    0          2m38s
pod/openmetadata-dependencies-web-88b974456-8zkh6             0/1     Pending    0          2m38s
pod/opensearch-0                                              0/1     Pending    0          2m38s

NAME                                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/kubernetes                      ClusterIP   10.60.128.1     <none>        443/TCP                      26h
service/mysql                           ClusterIP   10.60.132.171   <none>        3306/TCP                     2m41s
service/mysql-headless                  ClusterIP   None            <none>        3306/TCP                     2m41s
service/nfs-server                      ClusterIP   10.60.140.41    <none>        2049/TCP,20048/TCP,111/TCP   25h
service/openmetadata                    ClusterIP   10.60.136.61    <none>        8585/TCP,8586/TCP            108s
service/openmetadata-dependencies-web   ClusterIP   10.60.130.57    <none>        8080/TCP                     2m41s
service/opensearch                      ClusterIP   10.60.139.156   <none>        9200/TCP,9300/TCP            2m41s
service/opensearch-headless             ClusterIP   None            <none>        9200/TCP,9300/TCP,9600/TCP   2m41s

NAME                                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nfs-server                                1/1     1            1           25h
deployment.apps/openmetadata                              0/1     1            0           108s
deployment.apps/openmetadata-dependencies-db-migrations   0/1     1            0           2m40s
deployment.apps/openmetadata-dependencies-scheduler       0/1     1            0           2m40s
deployment.apps/openmetadata-dependencies-sync-users      0/1     1            0           2m40s
deployment.apps/openmetadata-dependencies-triggerer       0/1     1            0           2m40s
deployment.apps/openmetadata-dependencies-web             0/1     1            0           2m40s

NAME                                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/nfs-server-ff44c5f84                                1         1         1       25h
replicaset.apps/openmetadata-85b87d7b84                             1         1         0       109s
replicaset.apps/openmetadata-dependencies-db-migrations-54fd9ffd5   1         1         0       2m41s
replicaset.apps/openmetadata-dependencies-scheduler-695fb566f7      1         1         0       2m41s
replicaset.apps/openmetadata-dependencies-sync-users-7596cd6bb6     1         1         0       2m41s
replicaset.apps/openmetadata-dependencies-triggerer-5dbc4cb67f      1         1         0       2m41s
replicaset.apps/openmetadata-dependencies-web-88b974456             1         1         0       2m41s

NAME                          READY   AGE
statefulset.apps/mysql        0/1     2m41s
statefulset.apps/opensearch   0/1     2m41s

chiraggaur321@cloudshell:~ (complyshield)$ kubectl describe pod openmetadata-85b87d7b84-jdgsw
Name:             openmetadata-85b87d7b84-jdgswctl describe pod openmetadata-85b87d7b84-jdgsw
Namespace:        default
Priority:         0
Service Account:  openmetadata
Node:             gke-cluster-1-default-pool-b3e6b703-nxbw/10.128.0.16
Start Time:       Tue, 30 Apr 2024 17:28:51 +0000
Labels:           app.kubernetes.io/instance=openmetadata
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=openmetadata
                  app.kubernetes.io/version=1.3.3
                  helm.sh/chart=openmetadata-1.3.3
                  pod-template-hash=85b87d7b84
Annotations:      <none>
Status:           Pending
IP:               10.116.1.12
IPs:
  IP:           10.116.1.12
Controlled By:  ReplicaSet/openmetadata-85b87d7b84
Init Containers:
  run-db-migrations:
    Container ID:  containerd://a0fda33b74f476305dee7e325e969bb8e317656e9fb27aac7d5fccd748b4da70
    Image:         docker.getcollate.io/openmetadata/server:1.3.3
    Image ID:      docker.getcollate.io/openmetadata/server@sha256:dd5aa1348c8d25fb73b166dba2b4e4177fe14dbfcc4347e933260792ae78bedf
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/bash
      -c
      /opt/openmetadata/bootstrap/openmetadata-ops.sh migrate 
    State:          Running
      Started:      Tue, 30 Apr 2024 17:31:16 +0000
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Tue, 30 Apr 2024 17:28:52 +0000
      Finished:     Tue, 30 Apr 2024 17:31:15 +0000
    Ready:          False
    Restart Count:  1
    Environment Variables from:
      openmetadata-config-secret          Secret  Optional: false
      openmetadata-omd-secret             Secret  Optional: false
      openmetadata-db-secret              Secret  Optional: false
      openmetadata-search-secret          Secret  Optional: false
      openmetadata-authorizer-secret      Secret  Optional: false
      openmetadata-secretsmanager-secret  Secret  Optional: false
      openmetadata-web-secret             Secret  Optional: false
      openmetadata-authentication-secret  Secret  Optional: false
      openmetadata-eventmonitor-secret    Secret  Optional: false
      openmetadata-pipeline-secret        Secret  Optional: false
      openmetadata-jwt-secret             Secret  Optional: false
      openmetadata-fernetkey-secret       Secret  Optional: false
    Environment:
      DB_USER_PASSWORD:  <set to the key 'openmetadata-mysql-password' in secret 'mysql-secrets'>      Optional: false
      AIRFLOW_PASSWORD:  <set to the key 'openmetadata-airflow-password' in secret 'airflow-secrets'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gjl7d (ro)
Containers:
  openmetadata:
    Container ID:   
    Image:          docker.getcollate.io/openmetadata/server:1.3.3
    Image ID:       
    Ports:          8585/TCP, 8586/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Liveness:       http-get http://:http-admin/healthcheck delay=60s timeout=1s period=30s #success=1 #failure=5
    Readiness:      http-get http://:http/ delay=60s timeout=1s period=30s #success=1 #failure=5
    Startup:        http-get http://:http-admin/healthcheck delay=0s timeout=1s period=60s #success=1 #failure=5
    Environment Variables from:
      openmetadata-config-secret          Secret  Optional: false
      openmetadata-omd-secret             Secret  Optional: false
      openmetadata-db-secret              Secret  Optional: false
      openmetadata-search-secret          Secret  Optional: false
      openmetadata-authorizer-secret      Secret  Optional: false
      openmetadata-secretsmanager-secret  Secret  Optional: false
      openmetadata-web-secret             Secret  Optional: false
      openmetadata-authentication-secret  Secret  Optional: false
      openmetadata-eventmonitor-secret    Secret  Optional: false
      openmetadata-pipeline-secret        Secret  Optional: false
      openmetadata-jwt-secret             Secret  Optional: false
      openmetadata-fernetkey-secret       Secret  Optional: false
    Environment:
      DB_USER_PASSWORD:  <set to the key 'openmetadata-mysql-password' in secret 'mysql-secrets'>      Optional: false
      AIRFLOW_PASSWORD:  <set to the key 'openmetadata-airflow-password' in secret 'airflow-secrets'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gjl7d (ro)
Conditions:
  Type              Status
  Initialized       False 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  kube-api-access-gjl7d:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age                  From               Message
  ----    ------     ----                 ----               -------
  Normal  Scheduled  3m44s                default-scheduler  Successfully assigned default/openmetadata-85b87d7b84-jdgsw to gke-cluster-1-default-pool-b3e6b703-nxbw
  Normal  Pulled     3m42s                kubelet            Successfully pulled image "docker.getcollate.io/openmetadata/server:1.3.3" in 670ms (670ms including waiting)
  Normal  Pulling    79s (x2 over 3m42s)  kubelet            Pulling image "docker.getcollate.io/openmetadata/server:1.3.3"
  Normal  Created    78s (x2 over 3m42s)  kubelet            Created container run-db-migrations
  Normal  Started    78s (x2 over 3m42s)  kubelet            Started container run-db-migrations
  Normal  Pulled     78s                  kubelet            Successfully pulled image "docker.getcollate.io/openmetadata/server:1.3.3" in 608ms (608ms including waiting)
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$                  
chiraggaur321@cloudshell:~ (complyshield)$ kubectl logs pod/openmetadata-85b87d7b84-jdgsw

Defaulted container "openmetadata" out of: openmetadata, run-db-migrations (init)
Error from server (BadRequest): container "openmetadata" in pod "openmetadata-85b87d7b84-jdgsw" is waiting to start: PodInitializing
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$                      pod/openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7
error: there is no need to specify a resource type as a separate argument when passing arguments in resource/name form (e.g. 'kubectl get resource/<resource_name>' instead of 'kubectl get resourchiraggaur321@cloudshell:~ (complyshield)$ kubectl describe pod openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7

Name:             openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7
Namespace:        default
Priority:         0
Service Account:  airflow
Node:             <none>
Labels:           app=airflow
                  component=db-migrations
                  pod-template-hash=54fd9ffd5
                  release=openmetadata-dependencies
Annotations:      checksum/db-migrations-script: 37898f38b90abd06081105d992362ec9e0d0015123b69e758e59031a9e6ddfc9
                  checksum/secret-config-envs: 811c8dadc59dc5b71e2a49deeb21051889b0a5507e49f5d6f73bcf4b7d3e993b
                  checksum/secret-local-settings: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
                  cluster-autoscaler.kubernetes.io/safe-to-evict: true
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/openmetadata-dependencies-db-migrations-54fd9ffd5
Init Containers:
  check-db:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec timeout 60s airflow db check
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mgcxh (ro)
Containers:
  db-migrations:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      python
      -u
      /mnt/scripts/db_migrations.py
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /mnt/scripts from scripts (ro)
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mgcxh (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  dags-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-dags
    ReadOnly:   false
  logs-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-logs
    ReadOnly:   false
  scripts:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  openmetadata-dependencies-db-migrations
    Optional:    false
  kube-api-access-mgcxh:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  8m1s  default-scheduler  running PreBind plugin "VolumeBinding": Operation cannot be fulfilled on persistentvolumeclaims "openmetadata-dependencies-dags": the object has been modified; please apply your changes to the latest version and try again
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 

chiraggaur321@cloudshell:~ (complyshield)$ kubectl describe pod openmetadata-dependencies-scheduler-695fb566f7-kcl2g
Name:             openmetadata-dependencies-scheduler-695fb566f7-kcl2gtadata-dependencies-scheduler-695fb566f7-kcl2g
Namespace:        default
Priority:         0
Service Account:  airflow
Node:             <none>
Labels:           app=airflow
                  component=scheduler
                  pod-template-hash=695fb566f7
                  release=openmetadata-dependencies
Annotations:      checksum/config-pod-template: 7722c6af0d611108d40d4548f492fca151094f5984e90a955ef9aa3d3edef7df
                  checksum/secret-config-envs: 811c8dadc59dc5b71e2a49deeb21051889b0a5507e49f5d6f73bcf4b7d3e993b
                  checksum/secret-local-settings: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
                  cluster-autoscaler.kubernetes.io/safe-to-evict: true
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/openmetadata-dependencies-scheduler-695fb566f7
Init Containers:
  check-db:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec timeout 60s airflow db check
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zf9st (ro)
  wait-for-db-migrations:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec airflow db check-migrations -t 60
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zf9st (ro)
Containers:
  airflow-scheduler:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec airflow scheduler -n -1
    Liveness:  exec [/usr/bin/dumb-init -- /entrypoint python -Wignore -c import os
import sys

# suppress logs triggered from importing airflow packages
os.environ["AIRFLOW__LOGGING__LOGGING_LEVEL"] = "ERROR"

# shared imports
try:
    from airflow.jobs.job import Job
except ImportError:
    # `BaseJob` was renamed to `Job` in airflow 2.6.0
    from airflow.jobs.base_job import BaseJob as Job
from airflow.utils.db import create_session
from airflow.utils.net import get_hostname

# heartbeat check imports
try:
    from airflow.jobs.scheduler_job_runner import SchedulerJobRunner
except ImportError:
    # `SchedulerJob` is wrapped by `SchedulerJobRunner` since airflow 2.6.0
    from airflow.jobs.scheduler_job import SchedulerJob as SchedulerJobRunner

with create_session() as session:
    ########################
    # heartbeat check
    ########################
    # ensure the SchedulerJob with most recent heartbeat for this `hostname` is alive
    hostname = get_hostname()
    scheduler_job = session \
        .query(Job) \
        .filter_by(job_type=SchedulerJobRunner.job_type) \
        .filter_by(hostname=hostname) \
        .order_by(Job.latest_heartbeat.desc()) \
        .limit(1) \
        .first()
    if (scheduler_job is not None) and scheduler_job.is_alive():
        pass
    else:
        sys.exit(f"The SchedulerJob (id={scheduler_job.id}) for hostname '{hostname}' is not alive")
] delay=10s timeout=60s period=30s #success=1 #failure=5
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /opt/airflow/pod_templates/pod_template.yaml from pod-template (ro,path="pod_template.yaml")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zf9st (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  dags-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-dags
    ReadOnly:   false
  logs-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-logs
    ReadOnly:   false
  pod-template:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      openmetadata-dependencies-pod-template
    Optional:  false
  kube-api-access-zf9st:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  9m12s  default-scheduler  running PreBind plugin "VolumeBinding": Operation cannot be fulfilled on persistentvolumeclaims "openmetadata-dependencies-dags": the object has been modified; please apply your changes to the latest version and try again
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$                      
chiraggaur321@cloudshell:~ (complyshield)$ kubectl describe pod pod/openmetadata-dependencies-sync-users-7596cd6bb6-w622x
Name:             openmetadata-dependencies-sync-users-7596cd6bb6-w622x
Namespace:        default
Priority:         0
Service Account:  airflow
Node:             <none>
Labels:           app=airflow
                  component=sync-users
                  pod-template-hash=7596cd6bb6
                  release=openmetadata-dependencies
Annotations:      checksum/secret-config-envs: 811c8dadc59dc5b71e2a49deeb21051889b0a5507e49f5d6f73bcf4b7d3e993b
                  checksum/secret-local-settings: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
                  checksum/sync-users-script: 1107c40cce1588a76d882bfd67d8ad7005c6293721599343430b826bbfe95c5a
                  cluster-autoscaler.kubernetes.io/safe-to-evict: true
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/openmetadata-dependencies-sync-users-7596cd6bb6
Init Containers:
  check-db:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec timeout 60s airflow db check
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hhdk2 (ro)
  wait-for-db-migrations:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec airflow db check-migrations -t 60
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hhdk2 (ro)
Containers:
  sync-airflow-users:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      python
      -u
      /mnt/scripts/sync_users.py
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /mnt/scripts from scripts (ro)
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hhdk2 (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  dags-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-dags
    ReadOnly:   false
  logs-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-logs
    ReadOnly:   false
  scripts:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  openmetadata-dependencies-sync-users
    Optional:    false
  kube-api-access-hhdk2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  10m   default-scheduler  running PreBind plugin "VolumeBinding": Operation cannot be fulfilled on persistentvolumeclaims "openmetadata-dependencies-dags": the object has been modified; please apply your changes to the latest version and try again
  Warning  FailedScheduling  46s   default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ kubectl get pods
NAME                                                      READY   STATUS       RESTARTS        AGE
my-permission-pod                                         1/1     Running      0               25h
mysql-0                                                   0/1     Pending      0               11m
nfs-server-ff44c5f84-5hbnv                                1/1     Running      0               26h
openmetadata-85b87d7b84-jdgsw                             0/1     Init:Error   3 (2m51s ago)   10m
openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7   0/1     Pending      0               11m
openmetadata-dependencies-scheduler-695fb566f7-kcl2g      0/1     Pending      0               11m
openmetadata-dependencies-sync-users-7596cd6bb6-w622x     0/1     Pending      0               11m
openmetadata-dependencies-triggerer-5dbc4cb67f-wgxfr      0/1     Pending      0               11m
openmetadata-dependencies-web-88b974456-8zkh6             0/1     Pending      0               11m
opensearch-0                                              0/1     Pending      0               11m
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$                      openmetadata-dependencies-triggerer-5dbc4cb67f-wgxfr
Name:             openmetadata-dependencies-triggerer-5dbc4cb67f-wgxfrtadata-dependencies-triggerer-5dbc4cb67f-wgxfr
Namespace:        default
Priority:         0
Service Account:  airflow
Node:             <none>
Labels:           app=airflow
                  component=triggerer
                  pod-template-hash=5dbc4cb67f
                  release=openmetadata-dependencies
Annotations:      checksum/secret-config-envs: 811c8dadc59dc5b71e2a49deeb21051889b0a5507e49f5d6f73bcf4b7d3e993b
                  checksum/secret-local-settings: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
                  cluster-autoscaler.kubernetes.io/safe-to-evict: true
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/openmetadata-dependencies-triggerer-5dbc4cb67f
Init Containers:
  check-db:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec timeout 60s airflow db check
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5bhw8 (ro)
  wait-for-db-migrations:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec airflow db check-migrations -t 60
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5bhw8 (ro)
Containers:
  airflow-triggerer:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec airflow triggerer
    Liveness:  exec [/usr/bin/dumb-init -- /entrypoint python -Wignore -c import os
import sys

# suppress logs triggered from importing airflow packages
os.environ["AIRFLOW__LOGGING__LOGGING_LEVEL"] = "ERROR"

# shared imports
try:
    from airflow.jobs.job import Job
except ImportError:
    # `BaseJob` was renamed to `Job` in airflow 2.6.0
    from airflow.jobs.base_job import BaseJob as Job
from airflow.utils.db import create_session
from airflow.utils.net import get_hostname

# heartbeat check imports
try:
    from airflow.jobs.triggerer_job_runner import TriggererJobRunner
except ImportError:
    # `TriggererJob` is wrapped by `TriggererJobRunner` since airflow 2.6.0
    from airflow.jobs.triggerer_job import TriggererJob as TriggererJobRunner

with create_session() as session:
    # ensure the TriggererJob with most recent heartbeat for this `hostname` is alive
    hostname = get_hostname()
    triggerer_job = session \
        .query(Job) \
        .filter_by(job_type=TriggererJobRunner.job_type) \
        .filter_by(hostname=hostname) \
        .order_by(Job.latest_heartbeat.desc()) \
        .limit(1) \
        .first()
    if (triggerer_job is not None) and triggerer_job.is_alive():
        pass
    else:
        sys.exit(f"The TriggererJob (id={triggerer_job.id}) for hostname '{hostname}' is not alive")
] delay=10s timeout=60s period=30s #success=1 #failure=5
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5bhw8 (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  dags-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-dags
    ReadOnly:   false
  logs-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-logs
    ReadOnly:   false
  kube-api-access-5bhw8:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  82s   default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
chiraggaur321@cloudshell:~ (complyshield)$                      openmetadata-dependencies-web-88b974456-8zkh6
Name:             openmetadata-dependencies-web-88b974456-8zkh6 openmetadata-dependencies-web-88b974456-8zkh6
Namespace:        default
Priority:         0
Service Account:  airflow
Node:             <none>
Labels:           app=airflow
                  component=web
                  pod-template-hash=88b974456
                  release=openmetadata-dependencies
Annotations:      checksum/config-webserver-config: 8fe2e6d97286a3b0d6703e26df3984facf732fd9be071b5a21e4285d8b298181
                  checksum/secret-config-envs: 811c8dadc59dc5b71e2a49deeb21051889b0a5507e49f5d6f73bcf4b7d3e993b
                  checksum/secret-local-settings: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
                  cluster-autoscaler.kubernetes.io/safe-to-evict: true
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/openmetadata-dependencies-web-88b974456
Init Containers:
  check-db:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec timeout 60s airflow db check
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /opt/airflow/pod_templates/pod_template.yaml from pod-template (ro,path="pod_template.yaml")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-s22zx (ro)
  wait-for-db-migrations:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       <none>
    Host Port:  <none>
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec airflow db check-migrations -t 60
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /opt/airflow/pod_templates/pod_template.yaml from pod-template (ro,path="pod_template.yaml")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-s22zx (ro)
Containers:
  airflow-web:
    Image:      docker.getcollate.io/openmetadata/ingestion:1.3.3
    Port:       8080/TCP
    Host Port:  0/TCP
    Command:
      /usr/bin/dumb-init
      --
      /entrypoint
    Args:
      bash
      -c
      exec airflow webserver
    Liveness:   http-get http://:web/health delay=60s timeout=10s period=30s #success=1 #failure=10
    Readiness:  http-get http://:web/health delay=60s timeout=10s period=30s #success=1 #failure=10
    Environment Variables from:
      openmetadata-dependencies-config-envs  Secret  Optional: false
    Environment:
      DATABASE_PASSWORD:           <set to the key 'airflow-mysql-password' in secret 'airflow-mysql-secrets'>  Optional: false
      CONNECTION_CHECK_MAX_COUNT:  0
    Mounts:
      /opt/airflow/dags from dags-data (rw)
      /opt/airflow/logs from logs-data (rw)
      /opt/airflow/pod_templates/pod_template.yaml from pod-template (ro,path="pod_template.yaml")
      /opt/airflow/webserver_config.py from webserver-config (ro,path="webserver_config.py")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-s22zx (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  dags-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-dags
    ReadOnly:   false
  logs-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  openmetadata-dependencies-logs
    ReadOnly:   false
  pod-template:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      openmetadata-pod-template
    Optional:  false
  webserver-config:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  openmetadata-dependencies-webserver-config
    Optional:    false
  kube-api-access-s22zx:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  11m   default-scheduler  running PreBind plugin "VolumeBinding": Operation cannot be fulfilled on persistentvolumeclaims "openmetadata-dependencies-dags": the object has been modified; please apply your changes to the latest version and try again
  Warning  FailedScheduling  106s  default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ kubectl describe pod opensearch-0 
Name:             opensearch-0
Namespace:        default
Priority:         0
Service Account:  default
Node:             <none>
Labels:           app.kubernetes.io/component=opensearch
                  app.kubernetes.io/instance=openmetadata-dependencies
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=opensearch
                  app.kubernetes.io/version=2.7.0
                  apps.kubernetes.io/pod-index=0
                  controller-revision-hash=opensearch-9db4fc565
                  helm.sh/chart=opensearch-2.12.1
                  statefulset.kubernetes.io/pod-name=opensearch-0
Annotations:      configchecksum: e9acc41952a6fdae8a800efe4afe8b5d0df58d56f6f289d9744c7ce8f787644
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    StatefulSet/opensearch
Init Containers:
  fsgroup-volume:
    Image:      busybox:latest
    Port:       <none>
    Host Port:  <none>
    Command:
      sh
      -c
    Args:
      chown -R 1000:1000 /usr/share/opensearch/data
    Environment:  <none>
    Mounts:
      /usr/share/opensearch/data from opensearch (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zgr27 (ro)
Containers:
  opensearch:
    Image:       opensearchproject/opensearch:2.7.0
    Ports:       9200/TCP, 9300/TCP, 9600/TCP
    Host Ports:  0/TCP, 0/TCP, 0/TCP
    Limits:
      cpu:     2
      memory:  2048M
    Requests:
      cpu:      100m
      memory:   256M
    Readiness:  tcp-socket :9200 delay=0s timeout=3s period=5s #success=1 #failure=3
    Startup:    tcp-socket :9200 delay=5s timeout=3s period=10s #success=1 #failure=30
    Environment:
      node.name:             opensearch-0 (v1:metadata.name)
      discovery.seed_hosts:  opensearch-cluster-master-headless
      cluster.name:          opensearch
      network.host:          0.0.0.0
      OPENSEARCH_JAVA_OPTS:  -Xmx1g -Xms1g
      node.roles:            master,ingest,data,remote_cluster_client,
      discovery.type:        single-node
    Mounts:
      /usr/share/opensearch/config/opensearch.yml from config (rw,path="opensearch.yml")
      /usr/share/opensearch/data from opensearch (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zgr27 (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  opensearch:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  opensearch-opensearch-0
    ReadOnly:   false
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      opensearch-config
    Optional:  false
  kube-api-access-zgr27:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  2m25s  default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ kubectl get pods -n default openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7 -o jsonpath="{..image}"
docker.getcollate.io/openmetadata/ingestion:1.3.3 docker.getcollate.io/openmetadata/ingestion:1.3.3chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ kubectl get pods
NAME                                                      READY   STATUS     RESTARTS      AGE
my-permission-pod                                         1/1     Running    0             25h
mysql-0                                                   0/1     Pending    0             15m
nfs-server-ff44c5f84-5hbnv                                1/1     Running    0             26h
openmetadata-85b87d7b84-jdgsw                             0/1     Init:0/1   5 (83s ago)   14m
openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs7   0/1     Pending    0             15m
openmetadata-dependencies-scheduler-695fb566f7-kcl2g      0/1     Pending    0             15m
openmetadata-dependencies-sync-users-7596cd6bb6-w622x     0/1     Pending    0             15m
openmetadata-dependencies-triggerer-5dbc4cb67f-wgxfr      0/1     Pending    0             15m
openmetadata-dependencies-web-88b974456-8zkh6             0/1     Pending    0             15m
opensearch-0                                              0/1     Pending    0             15m
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 

chiraggaur321@cloudshell:~ (complyshield)$                  openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs --namespace default
Error from server (NotFound): pods "pod" not found logs pod openmetadata-dependencies-db-migrations-54fd9ffd5-8nvs
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ kubectl version
Client Version: v1.28.8
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.7-gke.1026000
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ helm version
version.BuildInfo{Version:"v3.9.3", GitCommit:"414ff28d4029ae8c8b05d62aa06c7fe3dee2bc58", GitTreeState:"clean", GoVersion:"go1.17.13"}
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ 
chiraggaur321@cloudshell:~ (complyshield)$ kubectl get pvc
NAME                             STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-mysql-0                     Pending                                      standard-rwo   27h
openmetadata-dependencies-dags   Pending                                      standard-rwo   47m
openmetadata-dependencies-logs   Pending                                      standard-rwo   47m
opensearch-opensearch-0          Pending                                      standard-rwo   27h
chiraggaur321@cloudshell:~ (complyshield)$

Custom Helm Values

No response

Have you joined Openmetadata Slack community?

Yes

[huw0]

I have hit the same problem trying to run the OpenMetadata locally using kind. Running kubectl get event showed that the underlying problem is:

11s         Warning   ProvisioningFailed                  persistentvolumeclaim/openmetadata-dependencies-dags            failed to provision volume with StorageClass "standard": Only support ReadWriteOnce access mode

At least for kind this issue is covered under kubernetes-sigs/kind#1487. The workaround described here enabled me to proceed.