WeKan Snap upgrade script, from 6.09 to newest

[xet7]

Hi, there is now upgrade script, that also migrates attachments and avatars.

At start of script is backup directory /var/wekanbackup , you may change it to directory where is enough disk space for copy of /var/snap/wekan/common and mongodump.

Check that you have enough disk space:

df -h

Check how much disk space /var/snap/wekan/common takes:

sudo du -sh /var/snap/wekan/common

If you need to attach external disk, that has more disk space for backup directory, some related info is at https://github.com/wekan/wekan/wiki/Repair-Docker

You can check what version you are running, with command:

sudo snap list

Example usage, at server, where is WeKan Snap 6.09:

sudo su

wget https://wekan.github.io/wekan-snap-upgrade.sh

chmod +x wekan-snap-upgrade.sh

./wekan-snap-upgrade.sh

Then, it will show menu:

Backups will be stored to /var/wekanbackup . Change it at top of this script, to where is enough disk space.
Backup directory requires a lot of disk space. It will have copy of /var/snap/wekan/common and mongodump.
1) Upgrade WeKan Snap from 6.09 to newest    3) Quit
2) Downgrade WeKan Snap from newest to 6.09
Please enter your choice:

When you write number 1 and press Enter, upgrading starts.

If you need to downgrade, write 2 and press Enter, downgrading starts.

If you write 3 and press Enter, it quits script.

Script will also show if your CPU does not support AVX, then at that old server new WeKan will not work, it requires moving WeKan to new server.

[xet7]

It looks like that to get the script to work, I had to manually first create backup directory:

sudo so

mkdir -p /var/wekanbackup

[Tanarri]

Hi, I had tried you upgrade script as mentioned in Post 1. It worked, but at the end I had following problems: a) My Avatar was missing - no problem. I uploaded it again b) the attachements are missing

A few Infos about a example attachment: The script says:

The filepath should be

The file ist missing:

The image should be here:

Any hints or can I help you with more informations?

[xet7]

Upgrade script only works if it can rename database tables properly, cfs to attachments and avatars tables. If there is already existing tables, attachments and avatars are not visible, table rename does not work.

I try to rewrite upgrade script to export/import attachments, something like at https://github.com/wekan/minio-metadata

[lezioul]

Hi, I had tried you upgrade script as mentioned in Post 1. It worked, but at the end I had following problems: a) My Avatar was missing - no problem. I uploaded it again b) the attachements are missing

A few Infos about a example attachment: The script says:

The filepath should be

The file ist missing:

The image should be here:

Any hints or can I help you with more informations?

Same experience here.

Upgrade script only works if it can rename database tables properly, cfs to attachments and avatars tables. If there is already existing tables, attachments and avatars are not visible, table rename does not work.

I try to rewrite upgrade script to export/import attachments, something like at https://github.com/wekan/minio-metadata

what do you mean by : "If there is already existing tables, attachments and avatars are not visible, table rename does not work."? if i upgrade from stable to candidate, tables will be already existing, so what's the workaround?

[xet7]

TLDR: Wait for newer updated upgrade script. Use current upgrade script option to downgrade back to WeKan 6.09, until new upgrade script is available.

Long answer:

Old tables, where name starts with text cfs, from WeKan 6.09, do exist. But if some previous upgrade attempt, or uploading new images with WeKan 7.x has created tables that start with Attachments and Avatars, those new should be deleted, so that renaming cfs tables works. This existing upgrade script did work for someone already, where attachments and avatars become visible.

Also, some metadata is probably missing from avatars, that needs to be copied also, similar like metadata etc was copied for attachments. Those differences can be seen with MongoDB GUI like Nosqlbooster or DBGate.

There are still improvements to be made, because some images may not be detected correctly as image. For example, some png and other images may need isImage: true so that at opened card image preview is available, for example:

let pngResult = db.attachments.updateMany(
  {
    name: /.png$/i,
    isImage: {$ne: true}
  },
  {
    $set: {
    isImage: true,
    type: "image/png",
    mime: "image/png",
    "mime-type": "image/png"
    }
  }
);

[xet7]

Also, currently I'm figuring out, how to get MongoDB 6.x version that does not require CPU to support AVX, so that it would work at all hardware. https://github.com/wekan/wekan/issues/4321#issuecomment-2467933444

[ChristianAlfredsson]

This might solve it @xet7. Please try

sudo snap run --shell wekan.mongodb mongosh --port 27019 use wekan

db.attachments.files.drop() db.attachments.chunks.drop() db.avatars.files.drop() db.avatars.chunks.drop()

db.cfs_gridfs.attachments.chunks.renameCollection('attachments.chunks') db.cfs_gridfs.attachments.files.renameCollection('attachments.files') db.cfs_gridfs.avatars.chunks.renameCollection('avatars.chunks') db.cfs_gridfs.avatars.files.renameCollection('avatars.files')

db.attachments.updateMany( { name: /.png$/i, isImage: {$ne: true} }, { $set: { isImage: true, type: "image/png", mime: "image/png", "mime-type": "image/png" } } );

(Add more image types if it is necesary)

[lezioul]

TLDR: Wait for newer updated upgrade script. Use current upgrade script option to downgrade back to WeKan 6.09, until new upgrade script is available.

Long answer:

Old tables, where name starts with text cfs, from WeKan 6.09, do exist. But if some previous upgrade attempt, or uploading new images with WeKan 7.x has created tables that start with Attachments and Avatars, those new should be deleted, so that renaming cfs tables works. This existing upgrade script did work for someone already, where attachments and avatars become visible.

Also, some metadata is probably missing from avatars, that needs to be copied also, similar like metadata etc was copied for attachments. Those differences can be seen with MongoDB GUI like Nosqlbooster or DBGate.

There are still improvements to be made, because some images may not be detected correctly as image. For example, some png and other images may need isImage: true so that at opened card image preview is available, for example:

let pngResult = db.attachments.updateMany(
  {
    name: /.png$/i,
    isImage: {$ne: true}
  },
  {
    $set: {
    isImage: true,
    type: "image/png",
    mime: "image/png",
    "mime-type": "image/png"
    }
  }
);

feel free to ask me to test, i've a complete test env on a VM i can flush anytime :)

[ChristianAlfredsson]

Try this @lezioul

1. Install 6.09
2. Add some cards with images and attachments. One image must be png
3. Upgrade with upgradescript
4. Runt this

sudo snap run --shell wekan.mongodb mongosh --port 27019 use wekan

db.attachments.files.drop() db.attachments.chunks.drop() db.avatars.files.drop() db.avatars.chunks.drop()

db.cfs_gridfs.attachments.chunks.renameCollection('attachments.chunks') db.cfs_gridfs.attachments.files.renameCollection('attachments.files') db.cfs_gridfs.avatars.chunks.renameCollection('avatars.chunks') db.cfs_gridfs.avatars.files.renameCollection('avatars.files')

db.attachments.updateMany( { name: /.png$/i, isImage: {$ne: true} }, { $set: { isImage: true, type: "image/png", mime: "image/png", "mime-type": "image/png" } } );

Is attachments and png-images working?

[lezioul]

What i've done :

1. i install a fresh 6.09
2. import my dump
3. add a new card, take a snapshot, save it as a png file and attached it to the card
4. launch wekan-snap-upgrade.sh as root
5. use your commands :

db.attachments.files.drop()
db.attachments.chunks.drop()
db.avatars.files.drop()
db.avatars.chunks.drop()

-> false (each)

db.cfs_gridfs.attachments.chunks.renameCollection('attachments.chunks')
db.cfs_gridfs.attachments.files.renameCollection('attachments.files')
db.cfs_gridfs.avatars.chunks.renameCollection('avatars.chunks')
db.cfs_gridfs.avatars.files.renameCollection('avatars.files')

-> { ok: 1 } (for each)

db.attachments.updateMany(
{
name: /.png$/i,
isImage: {$ne: true}
},
{
$set: {
isImage: true,
type: "image/png",
mime: "image/png",
"mime-type": "image/png"
}
}
);

-> { acknowledged: true, insertedId: null, matchedCount: 18, modifiedCount: 18, upsertedCount: 0 }

Results :

before update :

after update :

[ChristianAlfredsson]

Did you remeber to do this?

It looks like that to get the script to work, I had to manually first create backup directory:

sudo so

mkdir -p /var/wekanbackup

[lezioul]

of course i did, with the right commands ;)