arminfelder / gridfsmigrate

RocketChat GridFS to filesytem migration script
GNU General Public License v3.0
71 stars 23 forks source link
gridfs migration-tool rocketchat

RocketChat GridFS to FileSystem/AmazonS3 Migration

This is a script for migrating files uploaded to RocketChat from the default GridFS upload store to FileSystem/AmazonS3.

migrate -c [command] -d [s3 bucket|output directory] -r [dbname] -t [target]

Help

Run ./migrate.py -h to see all available options

Commands

Requirements

Dependencies

Environment

Steps

  1. Backup your MongoDB database so that you won't loose any data in case of any issues. (MongoDB Backup Methods)

  2. Change Storage Type in RocketChat under Administration> File Upload to FileSystem or AmazonS3. Update the relevant configuration under the corresponding head in configuration page.

  3. Start copying files to the new store

    • File System

      ./migrate.py -c dump -r rocketchat -t FileSystem -d ./uploads
    • S3

      ./migrate.py -c dump -r rocketchat -t AmazonS3 -d S3bucket_name
  4. Update the database to use new store (use -t AmazonS3 if you are migrating to S3)

    ./migrate.py -c updatedb -d /app/uploads -r rocketchat -t FileSystem
  5. Check if everything is working correctly. Ensure that there are no files missing.

  6. Remove obsolete data from GridFS

    ./migrate.py -c removeblobs -d /app/uploads -r rocketchat

Troubleshooting

On some configurations, it might help to add the parameters "directconnection=True and connect=False" to the MongoClient constructor, such as:

    MongoClient(..., retryWrites=False, directconnection=True, connect=False)[self.db]

So that the connection happens in Single topology.