vmware-tanzu / kubeapps

A web-based UI for deploying and managing applications in Kubernetes clusters
Other
5k stars 707 forks source link

kubeapps-postgresql-0 CrashLoopBackOff #8146

Open sjxuereb opened 4 days ago

sjxuereb commented 4 days ago

Describe the bug

Postgres keeps crashing after installation. I tried with a standard installation and also with an nfs storageclass. it keeps crashing on the simplest of installations/configuration. I tried installing kubeapps several times.

After logging in kubeapps dashboard the catalog takes a long time to load and then I notice the postgres pod crashes. The pod revives and crashes on every dashboard command.

kubeapps-postgresql-0 0/1 CrashLoopBackOff

OS: Ubuntu 24.04.1 LTS K8: Client Version: v1.31.3, Kustomize Version: v5.4.2, Server Version: v1.31.3 Helm: Version:"v3.16.3"

CHART NAME: kubeapps CHART VERSION: 17.0.3 APP VERSION: 2.12.0**

To Reproduce

helm install kubeapps oci://registry-1.docker.io/bitnamicharts/kubeapps --namespace kapps --set useHelm3=true --set postgresql.resourcesPreset=2xlarge

Control 10g Ram /8 cores/ 32G disk Workers 8g Ram /4cores/ 32G disk

Expected behavior

Normal behavior of the postgres and kubeapps dashboard.

Screenshots

Additional context I also tried with a storageclass. Same results, postgres keeps crashing.

--set global.defaultStorageClass=nfs-client \ --set postgresql.primary.persistence.enabled=true \ --set redis.master.persistence.enabled=true \ --set redis.replica.persistence.enabled=true

sjxuereb commented 4 days ago

I have since re-installed the chart as:

helm install kubeapps oci://registry-1.docker.io/bitnamicharts/kubeapps -n kubeapps --set global.defaultStorageClass=nfs-client --set frontend.service.type=LoadBalancer --set postgresql.primary.persistence.enabled=true --set postgresql.auth.postgresPassword=password --set postgresql.resourcesPreset=medium --set redis.master.persistence.enabled=true --set redis.auth.password=password --set redis.master.resourcesPreset=medium

The data folder has been created on the nfs and postgres runs for a few minutes. It then dies out of memory.

journalctl -k | grep 'Killed process' ( on the worker node gives ) : Memory cgroup out of memory: Killed process 32350 (postgres)

As an extra test I installed the postgres chart on its own and it worked with no issues. However the kubeapps embedded chart is failing with OOMKilled https://github.com/bitnami/charts/tree/main/bitnami/postgresql#parameters

Any help please. Thanks

singatias commented 1 day ago

I had the same issue following the guide to test it out

fea359 commented 1 day ago

Changing the StatefulSet of postgresql and increasing the upper limits of CPU and memory resources, for example, changing the CPU to 2 and the memory to 4096m, or directly changing to unlimited, can solve the problem.