search

eclipse-ditto / ditto

Eclipse Ditto™: Digital Twin framework of Eclipse IoT - main repository
https://eclipse.dev/ditto/
Eclipse Public License 2.0
696 stars 230 forks source link

Mongodb pod crashing for ditto version 3.3.6 #1726

Closed BHUVANESHWARI967 closed 1 year ago

BHUVANESHWARI967 commented 1 year ago

I'm trying to install ditto latest version using helm chart in minikube cluster, I have pulled the chart locally using, helm pull oci://registry-1.docker.io/eclipse/ditto

Then tried,

$ helm install -n ditto my-ditto ./ditto-3.3.6.tgz --wait
Error: INSTALLATION FAILED: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline

The mongodb pod has failed to restart continuously,

$ kubectl get pods -n ditto
NAME                                     READY   STATUS             RESTARTS   AGE
my-ditto-connectivity-546cb595cf-qvt2n   1/1     Running            0          12m
my-ditto-dittoui-5bf645c55f-cfjxs        1/1     Running            0          12m
my-ditto-gateway-68fb7fd975-hx726        1/1     Running            0          12m
my-ditto-mongodb-7458f7c79f-q48p8        0/1     CrashLoopBackOff   7          12m
my-ditto-nginx-9846686f5-sdhg7           1/1     Running            0          12m
my-ditto-policies-6844dc8dcc-dh9cc       1/1     Running            0          12m
my-ditto-swaggerui-549bc65fb5-5clgx      1/1     Running            0          12m
my-ditto-things-7d68fc5b94-jhcdr         1/1     Running            0          12m
my-ditto-thingssearch-55dcfdcd8d-dfg5s   1/1     Running            0          12m

The pod desc,

$ kubectl describe pod my-ditto-mongodb-7458f7c79f-q48p8 -n ditto
Name:             my-ditto-mongodb-7458f7c79f-q48p8
Namespace:        ditto
Priority:         0
Service Account:  my-ditto-mongodb
Node:             minikube/192.168.49.2
Start Time:       Thu, 24 Aug 2023 12:11:25 +0530
Labels:           app.kubernetes.io/component=mongodb
                  app.kubernetes.io/instance=my-ditto
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=mongodb
                  helm.sh/chart=mongodb-12.1.31
                  pod-template-hash=7458f7c79f
Annotations:      <none>
Status:           Running
IP:               172.17.0.5
IPs:
  IP:           172.17.0.5
Controlled By:  ReplicaSet/my-ditto-mongodb-7458f7c79f
Containers:
  mongodb:
    Container ID:   docker://932877cda6eed39ab2dcc535a502f6e7607a7409ee2d4ace2f68381b0df05c00
    Image:          docker.io/bitnami/mongodb:5.0.10-debian-11-r3
    Image ID:       docker-pullable://bitnami/mongodb@sha256:563e1572db6c23a7bc5d8970d4cf06de1f1a80bd41c4b5e273a92bfa9f26d0f1
    Port:           27017/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    132
      Started:      Thu, 24 Aug 2023 12:15:24 +0530
      Finished:     Thu, 24 Aug 2023 12:15:25 +0530
    Ready:          False
    Restart Count:  5
    Liveness:       exec [/bitnami/scripts/ping-mongodb.sh] delay=30s timeout=10s period=20s #success=1 #failure=6
    Readiness:      exec [/bitnami/scripts/readiness-probe.sh] delay=5s timeout=5s period=10s #success=1 #failure=6
    Environment:
      BITNAMI_DEBUG:                    false
      ALLOW_EMPTY_PASSWORD:             yes
      MONGODB_SYSTEM_LOG_VERBOSITY:     0
      MONGODB_DISABLE_SYSTEM_LOG:       no
      MONGODB_DISABLE_JAVASCRIPT:       no
      MONGODB_ENABLE_JOURNAL:           yes
      MONGODB_PORT_NUMBER:              27017
      MONGODB_ENABLE_IPV6:              no
      MONGODB_ENABLE_DIRECTORY_PER_DB:  no
    Mounts:
      /bitnami/mongodb from datadir (rw)
      /bitnami/scripts from common-scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from my-ditto-mongodb-token-f4zzs (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  common-scripts:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      my-ditto-mongodb-common-scripts
    Optional:  false
  datadir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  my-ditto-mongodb-token-f4zzs:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  my-ditto-mongodb-token-f4zzs
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  5m4s                   default-scheduler  Successfully assigned ditto/my-ditto-mongodb-7458f7c79f-q48p8 to minikube
  Warning  Unhealthy  4m34s (x2 over 4m43s)  kubelet            Readiness probe failed:
  Warning  Unhealthy  4m15s                  kubelet            Readiness probe failed: /opt/bitnami/scripts/libversion.sh: line 20: 1: version is required
/opt/bitnami/scripts/libversion.sh: line 20: 1: version is required
/opt/bitnami/scripts/libversion.sh: line 20: 1: version is required
  Normal   Created  3m16s (x4 over 4m59s)  kubelet  Created container mongodb
  Normal   Started  3m12s (x4 over 4m56s)  kubelet  Started container mongodb
  Warning  BackOff  2m41s (x9 over 4m12s)  kubelet  Back-off restarting failed container
  Normal   Pulled   2m29s (x5 over 5m)     kubelet  Container image "docker.io/bitnami/mongodb:5.0.10-debian-11-r3" already present on machine

The logs are,

$ kubectl logs -n ditto my-ditto-mongodb-7458f7c79f-q48p8 -c mongodb
mongodb 06:53:24.20 
mongodb 06:53:24.21 Welcome to the Bitnami mongodb container
mongodb 06:53:24.22 Subscribe to project updates by watching https://github.com/bitnami/containers
mongodb 06:53:24.22 Submit issues and feature requests at https://github.com/bitnami/containers/issues
mongodb 06:53:24.23 
mongodb 06:53:24.23 INFO  ==> ** Starting MongoDB setup **
mongodb 06:53:24.24 INFO  ==> Validating settings in MONGODB_* env vars...
mongodb 06:53:24.43 WARN  ==> You set the environment variable ALLOW_EMPTY_PASSWORD=yes. For safety reasons, do not use this flag in a production environment.
mongodb 06:53:24.46 INFO  ==> Initializing MongoDB...
mongodb 06:53:24.51 INFO  ==> Deploying MongoDB from scratch...
/opt/bitnami/scripts/libos.sh: line 336:    47 Illegal instruction     (core dumped) "$@" > /dev/null 2>&1

Could anyone please help me with this?

thjaeckle commented 1 year ago

Seems like a bitnami mongodb issue .. maybe this one? https://github.com/bitnami/charts/issues/12834

BHUVANESHWARI967 commented 1 year ago

Thank you for sharing. Yes, I think it's related to that. I'm encountering this same problem while installing cloud2edge package too.

https://github.com/bitnami/charts/issues/12834 This suggests downgrading the mongodb version, may I know which version is compatible with ditto and where should the changes be done?

thjaeckle commented 1 year ago

Supported MongoDB versions are documented here

BHUVANESHWARI967 commented 1 year ago

Tried making the change of version in mongodb chart, but getting the same error. Additionally added the bitnami repo and executed minikube tunnel. Is there any other file I need to make changes?

Can you please tell me what am I doing wrong?

thjaeckle commented 1 year ago

Can you please tell me what am I doing wrong?

No, you should ask MongoDB or Bitnami for MongoDB related issues