MongoDB CrashLoopBackOff Pod

[skarkarof]

Hello,

I'm following the Self-Hosted Installation Wiki for Kerberos Factory.

I'm using an Ubuntu 22.04 VPS I followed the entire step by step, but the MongoDB POD does not work.

mongodb mongodb-5b7545f4f-gc986 0/1 CrashLoopBackOff 13 (4m57s ago) 46m

See images below to check for errors. I've been racking my brain for hours and I can't figure it out.

kubectl describe pod mongodb-5b7545f4f-gc986 -n mongodb

Name: mongodb-5b7545f4f-gc986 Namespace: mongodb Priority: 0 Service Account: mongodb Node: nw-kerberos/10.158.0.11 Start Time: Tue, 05 Mar 2024 16:43:51 -0300 Labels: app.kubernetes.io/component=mongodb app.kubernetes.io/instance=mongodb app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=mongodb app.kubernetes.io/version=7.0.6 helm.sh/chart=mongodb-14.12.3 pod-template-hash=5b7545f4f Annotations: cni.projectcalico.org/containerID: c1834c039d7196a8338e476ff8803a31a4cc684a2847e28938553837acd71265 cni.projectcalico.org/podIP: 10.244.38.218/32 cni.projectcalico.org/podIPs: 10.244.38.218/32 Status: Running IP: 10.244.38.218 IPs: IP: 10.244.38.218 Controlled By: ReplicaSet/mongodb-5b7545f4f Containers: mongodb: Container ID: cri-o://8e4c050f500a049ddff098c25b6885e426655ce1ece907f43b320b04d25a4d75 Image: docker.io/bitnami/mongodb:4.4.15-debian-10-r8 Image ID: docker.io/bitnami/mongodb@sha256:916202d7af766dd88c2fff63bf711162c9d708ac7a3ffccd2aa812e3f03ae209 Port: 27017/TCP Host Port: 0/TCP SeccompProfile: RuntimeDefault State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 2 Started: Tue, 05 Mar 2024 17:35:27 -0300 Finished: Tue, 05 Mar 2024 17:35:27 -0300 Ready: False Restart Count: 15 Liveness: exec [/bitnami/scripts/ping-mongodb.sh] delay=30s timeout=10s period=20s #success=1 #failure=6 Readiness: exec [/bitnami/scripts/readiness-probe.sh] delay=5s timeout=5s period=10s #success=1 #failure=6 Environment: BITNAMI_DEBUG: false MONGODB_ROOT_USER: root MONGODB_ROOT_PASSWORD: <set to the key 'mongodb-root-password' in secret 'mongodb'> Optional: false ALLOW_EMPTY_PASSWORD: no MONGODB_SYSTEM_LOG_VERBOSITY: 0 MONGODB_DISABLE_SYSTEM_LOG: no MONGODB_DISABLE_JAVASCRIPT: no MONGODB_ENABLE_JOURNAL: yes MONGODB_PORT_NUMBER: 27017 MONGODB_ENABLE_IPV6: no MONGODB_ENABLE_DIRECTORY_PER_DB: no Mounts: /bitnami/mongodb from datadir (rw) /bitnami/scripts from common-scripts (rw) /opt/bitnami/mongodb/conf from empty-dir (rw,path="app-conf-dir") /opt/bitnami/mongodb/logs from empty-dir (rw,path="app-logs-dir") /opt/bitnami/mongodb/tmp from empty-dir (rw,path="app-tmp-dir") /tmp from empty-dir (rw,path="tmp-dir") Conditions: Type Status PodReadyToStartContainers True Initialized True Ready False ContainersReady False PodScheduled True Volumes: empty-dir: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium:
SizeLimit: common-scripts: Type: ConfigMap (a volume populated by a ConfigMap) Name: mongodb-common-scripts Optional: false datadir: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: mongodb ReadOnly: false QoS Class: BestEffort Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message

---

Normal Scheduled 54m default-scheduler Successfully assigned mongodb/mongodb-5b7545f4f-gc986 to nw-kerberos Normal Pulled 52m (x5 over 54m) kubelet Container image "docker.io/bitnami/mongodb:4.4.15-debian-10-r8" already present on machine Normal Created 52m (x5 over 54m) kubelet Created container mongodb Normal Started 52m (x5 over 54m) kubelet Started container mongodb Warning BackOff 4m (x244 over 54m) kubelet Back-off restarting failed container mongodb in pod mongodb-5b7545f4f-gc986_mongodb(fc6aa2b7-5e49-42cd-a94d-202a5e241260)

---

LOGS: root@nw-kerberos:/home/kuesttman/factory/kubernetes/mongodb# k logs pods/mongodb-5b7545f4f-gc986 -n mongodb mongodb 20:40:28.58 mongodb 20:40:28.59 Welcome to the Bitnami mongodb container mongodb 20:40:28.59 Subscribe to project updates by watching https://github.com/bitnami/containers mongodb 20:40:28.59 Submit issues and feature requests at https://github.com/bitnami/containers/issues mongodb 20:40:28.60 mongodb 20:40:28.60 INFO ==> Starting MongoDB setup mongodb 20:40:28.63 INFO ==> Validating settings in MONGODB_* env vars... mongodb 20:40:28.70 INFO ==> Initializing MongoDB... sed: can't read /opt/bitnami/mongodb/conf/mongodb.conf: No such file or directory

[skarkarof]

Can anyone help?

[oliviawindsir]

Hey I am also having some problem with my MongoDB pod, but rather different. Mine is in Pending state rather than CrashLoopBackOff. I have not reached any solutions for mine, but I thought maybe you should see if your load balancer service are all running well?

Will you be able to post your kubectl get all -A?

[oliviawindsir]

Hey I am also having some problem with my MongoDB pod, but rather different. Mine is in Pending state rather than CrashLoopBackOff. I have not reached any solutions for mine, but I thought maybe you should see if your load balancer service are all running well?

Will you be able to post your kubectl get all -A?

I ended up with the same issue and bumped into this ticket in bitnami repo here. However, I took another workaround which is to set up MongoDB atlas and create a free DB to point to. So I skipped setting up the mongoDB using helm chart provided by bitnami and just run the configmap set up to point to my hosted mongoDB. Now my system works.

Update : @skarkarof The above MongoDB Atlas won't work because kerberos factory is not able to resolve the IP of Atlas cluster. You will still need to create a MongoDB using bitnami which will result in one single IP so that kerberos factory can access it.

My working solution is to choose the release that is not affected by the issue reported in the issue logged in bitnami repo. I used helm install mongodb -n mongodb bitnami/mongodb --version 14.8.0 --values ./mongodb/values.yaml and it works. You will need to update mongodb.config.yaml to change the MONGODB_HOST to mongodb.mongodb.svc.cluster.local.

To check if it is successful, do a kubectl logs on your factory pods. You should see the following message :

Successfully Opened ./config/global.json                                                                                                                                                                    
Successfully Opened ./config/template.json                                                                                                                                                                  
{"level":"info","msg":"Running Kerberos Factory on :80","time":"2024-06-05T17:20:19+02:00"}