Incremental backup and restore

[brazilofmux]

Original issue 215 created by brazilofmux on 2006-11-15T21:51:09.000Z:

Purpose:

It would be useful if MUX had the ability to produce incremental backups, and perform incremental restores. In many cases where something is lost and must be reclaimed from backup, it is not sensible to do a 'rollback', but instead to extract the object or objects in question from a backup and manually restore them into the game. This can be a tedious process (less so thanks to Ashen-Shugar's dbpuller tool) and in the case of player objects it is difficult or impossible to restore them properly.

Additionally, older and larger games often produce backups of a significant size, requiring quite a lot of disk space to store them. It is likely the vast majority of the database is not altered on a frequent basis, and so substantial disk space could be reclaimed by using a full/incremental backup strategy.

Method:

The method for determining the scope of an incremental backup could be time/date based, in which a timestring is supplied to mark the boundary. For example:

@dump/incremental <timestring>

where <timestring> is in a MUX recognizable format, ie: Tue Nov 14 16:43:17 2006, or secs. Potentially, it might recognize a format such as - 5 meaning anything modified in the past 5 days. The MUX would then follow its normal process for writing a flatfile to disk, except that it would skip objects whose age (via 'modified' attribute) was older than the specified period allowed for.

The method of incremental restores is likely more complex. It might simply require a script that can merge changes from an incremental flatfile to a full flatfile. There might be more elegant solutions than that.

[brazilofmux]

Comment #1 originally posted by brazilofmux on 2006-11-17T06:03:10.000Z:

<empty>

[brazilofmux]

Comment #2 originally posted by brazilofmux on 2006-12-01T05:59:58.000Z:

<empty>

[brazilofmux]

Comment #3 originally posted by brazilofmux on 2007-08-25T20:52:12.000Z:

As MUX continues to mature, games using MUX become older and their databases expand. I think this feature will only become more valuable as time goes on.

[brazilofmux]

Comment #4 originally posted by brazilofmux on 2007-09-21T00:48:30.000Z:

This might benefit from a module implementation.

[brazilofmux]

Comment #5 originally posted by brazilofmux on 2009-02-28T15:40:58.000Z:

A thought: Adding an optional (config param?) cache that keeps track of objects that have been modified since last cache reset. This could be used to speed up discovery of objects as opposed to checking 'Modified' on everything in the database.

Probably only useful on larger games, where it would become more expensive to maintain. However, larger games probably have a greater number of unmodified objects, which might mitigate the cost.

Cache could be reset during a normal full backup, or optionally during incrementals to make them cumulative incrementals.