search

cjmling / findings

Notes on stuff i finds worth keeping for quick reference later on.
2 stars 0 forks source link

Mongodb changing storage engine from mmapv1 to wiredTiger #149

Open cjmling opened 5 years ago

cjmling commented 5 years ago

Ref: https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781787126480/1/ch01lvl1sec15/changing-storage-engine

https://docs.mongodb.com/manual/tutorial/change-standalone-wiredtiger/

Note quick explain:

  1. Create backup of existing database into a new folder A
  2. Create an empty folder B
  3. Start a new mongo process with cofig dbPath as folder B and engine as wiredTiger
  4. Mongorestore the database from folder A . The restored database with wiredTiger as storage engine will be in folder B.

Thats it now you have your mongo database as wiredTiger stored in folder B. But because its a new folder we just created. When you restart mongo in future it might not know about this new location.

So you got to check/set the configuration file of mongo to point dbPath to this new location. In debian its located at /etc/mongod.conf. Also the engine if its set to mmapv1.

Or you can just rename the original database folder , which most likely located at /data/db or /data/mongo or /mnt/data/to something with-old` suffix. And change the just created folder B to same as original database folder name.

cjmling commented 5 years ago

FR , Chef , cookbook , service related specific instructions. May differ from system to system setup

Changing storage engine from mmapv1 to wiredTiger

  1. Make sure if we are using mmapv1 storage enginemongo --eval 'db.serverStatus().storageEngine' 

  2. Create backup of current mongodb mkdir /mnt/data/mongodb-backup mongodump --out /mnt/data/mongodb-backup

  3. Stop mongod service

service mongod stop

  1. Create empty directory, which will hold new db with wiredTiger storage engine mkdir /mnt/data/mongodbwt

  2. Edit /etc/mongod.conf 's dbPath and engine. Do it manually or automatically. 

Manually using vim command: change 2 line dbPath: "/mnt/data/mongodb" ===> dbPath: "/mnt/data/mongodbwt" engine: mmapv1 ===> engine: wiredTiger

Automatically using sed command: sed 's+data/mongodb+data/mongodbwt+g' /etc/mongod.conf sed -i 's/mmapv1/wiredTiger/g' /etc/mongod.conf

  1. Start mongod service

service mongod start

  1. Restore the backup mongodb ( this will get restore in mongodbwt folder )

mongorestore --noIndexRestore /mnt/data/mongodb-backup

Using -noIndexRestore because since v3.4 , mongodb added a validation on indexes specification. Already existing index like safe=true is not a valid index specification. So it will throw error. NOTE: To be specific this happen in footyroom.people collection.

NOTE: With --noIndexRestore , later we can re-index again using script index-models.js right ?

  1. At this stage we should already have working website with new mongodb storage engine. But because we just changed the dbPath and engine manually, we need to change this in cookbook too.

8.1 engine: as wiredTiger storage engine is going to be default engine. We may choose to just remove storage engine config from attributes/default.rb

8.2 dbPath: We seems to have two option here change the dbPath in kitchen.yml to be /mnt/data/mongodbwt delete old /mnt/data/mongodb folder and rename mongodbwt to mongodb

YAY! WE ARE DONE