search

bitnami / charts

Bitnami Helm Charts
https://bitnami.com
Other
8.92k stars 9.18k forks source link

[bitnami/mongodb] MongoDB replicaset fails when "primary" is in the release name #29821

Closed Lygoria closed 4 days ago

Lygoria commented 5 days ago

Name and Version

bitnami/mongodb 15.0.0 to 16.0.0 (at least)

What architecture are you using?

None

What steps will reproduce the bug?

Deploy MongoDB charts with replicaset and a release-name that contains primary:

Command to reproduce:

helm install \
  --namespace=mongo \
  --create-namespace \
  mongo-primary \
  oci://registry-1.docker.io/bitnamicharts/mongodb \
  --set="arbiter.enabled=false" \
  --set="architecture=replicaset" \
  --set="replicaCount=3" \
  --set="persistence.enabled=true" \
  --set="global.storageClass=local-path" \
  --set="auth.enabled=false"

Are you using any custom parameters or values?

architecture: replicaset
replicaCount: 3
auth:
  enabled: false
global:
  storageClass: local-path
persistence:
  enabled: true
arbiter:
  enabled: false

What is the expected behavior?

All members of the replicaset starts correctly

What do you see instead?

The second (or sometimes third, it seems random) pod fails. The pod starts with the following logs:

$ kubectl -n mongo logs --previous mongo-primary-mongodb-1
Defaulted container "mongodb" out of: mongodb, log-dir (init)
mongodb 15:02:27.57 INFO  ==> Advertised Hostname: mongo-primary-mongodb-1.mongo-primary-mongodb-headless.mongo.svc.cluster.local
mongodb 15:02:27.57 INFO  ==> Advertised Port: 27017
realpath: /bitnami/mongodb/data/db: No such file or directory
mongodb 15:02:27.58 INFO  ==> Data dir empty, checking if the replica set already exists
mongodb 15:02:28.43 INFO  ==> Detected existing primary: mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongodb 15:02:28.43 INFO  ==> Current primary is different from this node. Configuring the node as replica of mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongodb 15:02:28.44 INFO  ==> 
mongodb 15:02:28.44 INFO  ==> Welcome to the Bitnami mongodb container
mongodb 15:02:28.44 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
mongodb 15:02:28.44 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
mongodb 15:02:28.45 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
mongodb 15:02:28.45 INFO  ==> 
mongodb 15:02:28.45 INFO  ==> ** Starting MongoDB setup **
mongodb 15:02:28.50 INFO  ==> Validating settings in MONGODB_* env vars...
mongodb 15:02:28.55 INFO  ==> Initializing MongoDB...
mongodb 15:02:28.64 INFO  ==> Writing keyfile for replica set authentication...
mongodb 15:02:28.65 INFO  ==> Deploying MongoDB from scratch...
MongoNetworkError: connect ECONNREFUSED 192.168.78.34:27017
mongodb 15:02:30.11 INFO  ==> Creating users...
mongodb 15:02:30.11 INFO  ==> Users created
mongodb 15:02:30.12 INFO  ==> Enabling authentication...
mongodb 15:02:30.19 INFO  ==> Configuring MongoDB replica set...
mongodb 15:02:30.19 INFO  ==> Stopping MongoDB...
mongodb 15:02:33.81 INFO  ==> Trying to connect to MongoDB server mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local...
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
mongodb 15:03:08.84 ERROR ==> Unable to connect to host mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local
mongodb 15:03:08.85 INFO  ==> Stopping MongoDB...

The log the required argument 'port' was not provided is strange as there is a port configured. Then the pod restarts and stays in error (Readiness probe failed: Error: Not ready) apparently due to an incomplete configuration:

$ kubectl -n mongo logs  mongo-primary-mongodb-1 | head -n 100
Defaulted container "mongodb" out of: mongodb, log-dir (init)
mongodb 15:03:10.73 INFO  ==> Advertised Hostname: mongo-primary-mongodb-1.mongo-primary-mongodb-headless.mongo.svc.cluster.local
mongodb 15:03:10.73 INFO  ==> Advertised Port: 27017
mongodb 15:03:10.73 INFO  ==> Pod name doesn't match initial primary pod name, configuring node as a secondary
mongodb 15:03:10.74 INFO  ==> 
mongodb 15:03:10.75 INFO  ==> Welcome to the Bitnami mongodb container
mongodb 15:03:10.75 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
mongodb 15:03:10.75 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
mongodb 15:03:10.75 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
mongodb 15:03:10.75 INFO  ==> 
mongodb 15:03:10.75 INFO  ==> ** Starting MongoDB setup **
mongodb 15:03:10.77 INFO  ==> Validating settings in MONGODB_* env vars...
mongodb 15:03:10.84 INFO  ==> Initializing MongoDB...
mongodb 15:03:10.94 INFO  ==> Writing keyfile for replica set authentication...
mongodb 15:03:11.00 INFO  ==> Deploying MongoDB with persisted data...
mongodb 15:03:11.02 INFO  ==> ** MongoDB setup finished! **
mongodb 15:03:11.03 INFO  ==> ** Starting MongoDB **
{"t":{"$date":"2024-10-08T15:03:11.109Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"enableLocalhostAuthBypass":{"default":true,"value":false}}}}
{"t":{"$date":"2024-10-08T15:03:11.148+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"main","msg":"***** SERVER RESTARTED *****"}

[ ...]

{"t":{"$date":"2024-10-08T15:03:13.000+00:00"},"s":"W",  "c":"QUERY",    "id":23799,   "ctx":"ftdc","msg":"Aggregate command executor error","attr":{"error":{"code":26,"codeName":"NamespaceNotFound","errmsg":"Unable to retrieve storageStats in $collStats stage :: caused by :: Collection [local.oplog.rs] not found."},"stats":{},"cmd":{"aggregate":"oplog.rs","cursor":{},"pipeline":[{"$collStats":{"storageStats":{"waitForLock":false,"numericOnly":true}}}],"$db":"local"}}}

Additional information

Replicaset is KO:

> rs.conf()
MongoServerError[NotYetInitialized]: no replset config has been received
carrodher commented 4 days ago

Thank you for opening this issue and submitting the associated Pull Request. Our team will review and provide feedback. Once the PR is merged, the issue will automatically close.

Your contribution is greatly appreciated!