Incremental Backup on Multiple Databases

[ubiquitousthey]

I am working on a MongoDB implementation, and I am trying to use your tools. They are exactly what I am looking for, but I have having a problem. I need to backup multiple databases.

This creates a problem in the stopping for the oplog monitor. Touching the stop.txt file in the CWD doesn't work that well since I really need to be able to stop the oplog monitor for individual databases. I would even settle for stopping all, but since it deletes the stop file after it stops, I would have to keep adding a file and checking for whether or not it deletes it within a second or so to see if there are any more left.

It seems like it would be better if it responded to signals or if the stop file could go in the output directory.

[fehguy]

Hi, you might try modifying the StopFileMonitor.java file to pick up the stop file in any location--the alternative solutions are:

1) run multiple instances of the incremental backup util

2) hack the util to be more intelligent, by skipping the namespace of databases that you don't want to process. Take a look at this file:

https://github.com/wordnik/wordnik-oss/blob/master/modules/mongo-admin-utils/src/main/java/com/wordnik/system/mongodb/OplogTailThread.java#L217

Where you can easily override the logic for determining if a record (which identifies the souce database as well) should be written.

[ubiquitousthey]

Thanks. For the time being I modified the CLASSPATH in the launching script to use a relative path so that I could launch it from the destination directory. I write the stop.txt file there.

Your second suggestion is a good one as well. I'll take a look at forking and creating a PR to have just one monitoring process. That would be good for new databases as well.

[fehguy]

Great, the code should be very modular and easy to change