Failed to Update RocketChat Helm Chart

david9991 commented 1 year ago

Description

I attempted to update the RocketChat Helm chart from version 5.3.2 to 5.4.3, but I encountered an error message.

Error Message

The error message I received was:

rocketchat 0s Warning FailedMount pod/rocketchat-rocketchat-pre-upgrade MountVolume.SetUp failed for volume "mongodb-script" : configmap "rocketchat-rocketchat-scripts" not found
rocketchat                   0s          Warning   FailedMount          pod/rocketchat-rocketchat-pre-upgrade                Unable to attach or mount volumes: unmounted volumes=[mongodb-script], unattached volumes=[mongodb-script kube-api-access-8sqjl]: timed out waiting for the condition

Previous Version

In the previous version (v5.3.2), there were two configmaps available: rocketchat-mongodb-fix-clustermonitor-... and rocketchat-mongodb-scripts. There was no rocketchat-rocketchat-scripts.

Expected Behavior

I expected the update to proceed successfully without any errors.

Actual Behavior

The update failed due to the error message above.

Steps to Reproduce

Attempt to update the RocketChat Helm chart from version 5.3.2 to 5.4.3.
Observe the error message.

Possible Solution

It appears that the rocketchat-rocketchat-scripts configmap is missing. It may need to be added or updated in order for the update to proceed successfully.

Environment

RocketChat Helm chart v5.3.2
Attempting to update to v5.4.3

masikrus commented 1 year ago

same problem

masikrus commented 1 year ago

Oh... there the solutions https://github.com/RocketChat/helm-charts/tree/master/rocketchat readme says "you need upgrade mongo your self"

just do kubectl -n $NAMESPACE edit statefulsets.apps rocketchat-mongodb and set first image tag 5.0.15-debian-11-r7 after pod start look to logs and wait upgrade mongo to 5.0 then exec to mongo container and set compatible versions kubectl -n $NAMESPACE exec -it rocketchat-mongodb-0 -- bash mongo -u $USER -p $PASSWORD db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } ) then again edit statefulsets but now set tag 6.0.5-debian-11-r0 and then again logs, wait upgrade, and set compatible version to 6.0 mongosh -u $USER -p $PASSWORD db.adminCommand( { setFeatureCompatibilityVersion: "6.0" } ) and then apply file rocketchat-rocketchat-scripts

apiVersion: v1
kind: ConfigMap
metadata:
  name: "rocketchat-rocketchat-scripts"
  annotations:
    meta.helm.sh/release-name: rocketchat
    meta.helm.sh/release-namespace: stage
  labels:
    app.kubernetes.io/name: rocketchat
    helm.sh/chart: rocketchat-5.3.2
    app.kubernetes.io/instance: rocketchat
    app.kubernetes.io/managed-by: Helm

data:
  verifyMongodb.js: |
    const versions = [
      "4.0",
      "4.2",
      "4.4",
      "5.0",
      "6.0"
    ];
    // [0]=mongosh path, [1]=mongosh path, [2]=connection string, [3]=script path, [4]=new tag
    const newVersion = process.argv[4].split('.').splice(0, 2).join('.');
    const currentVersion = db.version().split('.').splice(0, 2).join('.');
    if (currentVersion === newVersion) quit(0);
    const featureCompatibilityVersionResp = db.adminCommand({ getParameter:
      1, featureCompatibilityVersion: 1 });
    if (!featureCompatibilityVersionResp.ok) {
      print("failed to read current feature compatibility version");
      quit(1);
    }
    const featureCompatibilityVersion = featureCompatibilityVersionResp.featureCompatibilityVersion.version;
    if (featureCompatibilityVersion !== currentVersion) {
      print("current feature compatibility version is incompatible with the mongodb version you're trying to update to");
      quit(1);
    }
    if (versions.indexOf(newVersion) - versions.indexOf(currentVersion) !== 1) {
      print("you can't skip a mongodb version while updating, please read our update documentation for more information");
      quit(1);
    }
    quit(0);

just install by helm -n $NAMESPACE apply -f $YML_FILE

And THEN! ) edit file values.yml, set mongodb: image: tag: 6.0.5-debian-11-r0 And then you can upgrade to release v5.4.3 without problem

P.S.

edit file rocketchat-rocketchat-scripts by you namespace
on mongodb 6.0+ cli binary named mongosh
maybe after upgrade need scale mongo statefulsets from 1 to 0 and from 0 to 1 kubectl -n stage scale --replicas=0 statefulset rocketchat-mongodb

david9991 commented 1 year ago

To resolve the issue, I performed the following steps:

Dumped the old database
Performed a clean reinstallation
Restored the database from the dump to the new version

After completing these steps, my problem was resolved.

david9991 commented 1 year ago

I believe it's the same issue as described in https://github.com/RocketChat/helm-charts/issues/93.

masikrus commented 1 year ago

To resolve the issue, I performed the following steps:

Dumped the old database

Performed a clean reinstallation

Restored the database from the dump to the new version

After completing these steps, my problem was resolved.

You can't dump mongo 4.4 and restore to 5 or 6 version, dump and restore in mongodb work only in version to version

alexkorotysh commented 1 year ago

I have the same issue, looks like pod XXXXXX-pre-upgrade doesn't have configmap with XXXXXX-scripts before the upgrade.

Davidffry commented 1 year ago

Hello, how I upgrade RocketChat :

Previous version :

rocketchat : v5.3.4
mongodb : v4.4.11

Steps :
Creation of ConfigMap : rocket-chat-rocketchat-scripts (enable to check the version to migration) - kubectl apply -f rocket-chat-rocketchat-scripts.yml
Upgrade mongoDB statefulSet to 5.0 version manually
Exec in MongoDB : db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )
- Into the MongoDB pod run : mongosh -u root -p
- Enter the password from the secret
- Run this above command
Apply Helm chart for 5.4.3 (Normally we can apply to 6.4.3 but for security reason upload to 5.4.3 before)
Check if it work
It does not work : Rollback, re-run test.
If works,
Apply Helm chart for 6.3.4
Exec in MongoDB : db.adminCommand( { setFeatureCompatibilityVersion: "6.0" } )
- Into the MongoDB pod run : mongosh -u root -p
- Enter the password from the secret
- Run to check : db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
- Run this above command
Check if it work

#rocket-chat-rocketchat-scripts.yml
apiVersion: v1
kind: ConfigMap
metadata:
  name: rocket-chat-rocketchat-scripts
  namespace: rocketchat
  labels:
    app.kubernetes.io/instance: rocket-chat
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: rocketchat
    helm.sh/chart: rocketchat-6.3.4
    k8slens-edit-resource-version: v1
  annotations:
    meta.helm.sh/release-name: rocket-chat
    meta.helm.sh/release-namespace: rocketchat
  selfLink: /api/v1/namespaces/rocketchat/configmaps/rocket-chat-rocketchat-scripts
data:
  verifyMongodb.js: >
    const versions = [
      "4.0",
      "4.2",
      "4.4",
      "5.0",
      "6.0"
    ];

    // [0]=mongosh path, [1]=mongosh path, [2]=connection string, [3]=script
    path, [4]=new tag

    const newVersion = process.argv[4].split('.').splice(0, 2).join('.');

    const currentVersion = db.version().split('.').splice(0, 2).join('.');

    if (currentVersion === newVersion) quit(0);

    const featureCompatibilityVersionResp = db.adminCommand({ getParameter:
      1, featureCompatibilityVersion: 1 });
    if (!featureCompatibilityVersionResp.ok) {
      print("failed to read current feature compatibility version");
      quit(1);
    }

    const featureCompatibilityVersion =
    featureCompatibilityVersionResp.featureCompatibilityVersion.version;

    if (featureCompatibilityVersion !== currentVersion) {
      print("current feature compatibility version is incompatible with the mongodb version you're trying to update to");
      quit(1);
    }

    if (versions.indexOf(newVersion) - versions.indexOf(currentVersion) !== 1) {
      print("you can't skip a mongodb version while updating, please read our update documentation for more information");
      quit(1);
    }

    quit(0);

Targetd Version :

rocketchat : v6.3.4
mongodb : v6.0XXXX

RocketChat / helm-charts