Backups should be verifed

[bhavicp]

When backing up, backups should be verifed.

• md5 sum could be checked.

[hamstar]

We can do this using find and md5sum for example:

root@braincase:# find /home/test/test.git -type f -exec md5sum {} \;
41adefcd1f91e91d473275a100d31a2c  /home/test/test.git/config
a721c621608f7481d9cb4c505ad83b34  /home/test/test.git/refs/heads/master
a0a7c3fff21f2aea3cfa1d0316dd816c  /home/test/test.git/description
2b7ea5cee3c49ff53d41e00785eb974c  /home/test/test.git/hooks/post-update.sample
a4a7e457b55b5ac2877f7973dbba37e9  /home/test/test.git/hooks/pre-applypatch.sample
3ff6ba9cf6d8e5332978e057559b5562  /home/test/test.git/hooks/pre-rebase.sample
98d7f8e291edbab8df79e433dc97ecc8  /home/test/test.git/hooks/update.sample
579a3c1e12a1e74a98169175fb913012  /home/test/test.git/hooks/commit-msg.sample
03bdf3259f84884dc51b8eceb16956c9  /home/test/test.git/hooks/post-receive.example
7dfe15854212a30f346da5255c1d794b  /home/test/test.git/hooks/prepare-commit-msg.sample
9cc72dc973e24f9623bd3fe708f60ef5  /home/test/test.git/hooks/applypatch-msg.sample
15449d98cfa79704332d057b3f91093c  /home/test/test.git/hooks/pre-commit.sample
496d4cc726c82496850b55cbd51d8e8e  /home/test/test.git/objects/18/c48297652174ffae65b877dd131711a5746181
ab5f20dcd5b558888db7d80b0f979f8a  /home/test/test.git/objects/d5/64d0bc3dd917926892c55e3706cc116d5b165e
ce8157901d8116e7cddfd5e7ea4dcf26  /home/test/test.git/objects/d1/7b3f6e22960eac24831317b8df12d56742afa8
12875884d80f5ef256816dff7083ba33  /home/test/test.git/objects/b1/f187c2e9acaba942639bca90a63c5b4f058967
30620df6a655176b6c5e1b1a232d6489  /home/test/test.git/objects/c8/8b2a15165058d5921a3ffd7cdffe9fd5c73f48
a83a3443eceb9883bbe55e791ae7a651  /home/test/test.git/objects/c9/db4591825bd7a918df686ff04aeb3a87d3bda0
a4619d0453ea53d85d3e54a1aef42c2b  /home/test/test.git/objects/b5/8d1184a9d43a39c0d95f32453efc78581877d6
6ed65aa6a9ac54b8629139c24fe2aca3  /home/test/test.git/objects/4f/31f00e4639a04a6dfd042fdc2263c12b6548a6
f653c0ab06e6663638aeffbe233c68a8  /home/test/test.git/objects/ec/17ec1939b7c3e86b7cb6c0c4de6b0818a7e75e
22155efed4397dada220e5e28997899e  /home/test/test.git/objects/07/d359d07cf1ed0c0074fdad71ffff5942f0adfa
3f312709086b812d7ccc615f74e887bf  /home/test/test.git/objects/8b/2a2fe84feaeaba56953d6d4d0d649b3cf755eb
dc5a81f09c45d0c2a9f1fcd46546d422  /home/test/test.git/objects/49/8b267a8c7812490d6479839c5577eaaec79d62
4ab3af9e168d93d4f96f4eb08a25a3d7  /home/test/test.git/objects/e2/37605960070bfe48e180b807bc8d938fec1977
95068c2ea414ef36143e2202e3dd6d88  /home/test/test.git/objects/de/4f5f75d7ccd3a5b62bd2ce683ed678a5cb72c2
76cef38413e5b6746dbb20dbe402b116  /home/test/test.git/objects/25/d8397442f49f2fbbb33b82a6cedd64f19b6173
760c17fc00b5883271267c8dbf551eea  /home/test/test.git/objects/b3/33a637499bfad87ed525cdafc4dd3cdf65aebb
d6b87cc3b7af1feab28d9344e1a21950  /home/test/test.git/objects/f0/93a02ec49918ab15e920f455979fd5ed732cf6
bdd1bd117bb459b740cf47880d071c05  /home/test/test.git/objects/a0/fa0a0359dbb5f0755cffa54d7b5b14923a1bad
8374dff94cc2712df9e03766e1bdab62  /home/test/test.git/objects/a5/196d1be8fb59edf8062bef36d3a602e0812139
9b3a8475815c4a21dbb8dca1093602e9  /home/test/test.git/objects/e9/6b37fb46a87ddb6f6d624246c477ea3ac8ff3d
6d2d6d9c5be91cc65040a602060a8293  /home/test/test.git/objects/cb/089cd89a7d7686d284d8761201649346b5aa1c
b6df5ca3c856df73216200bf9f947562  /home/test/test.git/objects/33/730ca647cb2f7d77c64a6db08b362c18ed8464
c70c34cbeefd40e7c0149b7a0c2c64c2  /home/test/test.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391
f3c06017cd533e839c79e6b627fa9925  /home/test/test.git/objects/71/ab04edc09be7aeefa1e8a0f609a974ffd55a9f
ffdd306f1334b66d322a53ba140da0a6  /home/test/test.git/objects/68/24fa004152dbc597a9540f27852ec4dd39e42d
4cf2d64e44205fe628ddd534e1151b58  /home/test/test.git/HEAD
036208b4a1ab4a235d75c181e685e5a3  /home/test/test.git/info/exclude

Do a find op on the just backed up stuff and the current stuff and look for discrepancies. This is somewhere when locking might come in handy as any wayward edits could give false positives.

[hamstar]

This has been further clarified by James in the 12th Oct meeting.

We should have a few levels of validation.

[hamstar]

Level 1

This level will compare the md5sums of two backup archives.

This needs to be able to be run from the braincase-backup script so that the archive can be verified after the backup.

The client should be able to:

1. Click on a timestamp on the memories page
2. See a list of backups of that timestamp, the sources, and the md5sums of each

[hamstar]

Level 2

This level will compare the md5sums of the files between two backup archives. This will output the md5sum files which are different in the archives.

The client should be able to:

1. Be on the memories page
2. Select two backups to compare
3. Click compare
4. See what files have changed or have been added or removed in the newest archive

[hamstar]

Level 3

This level will do a diff between specific files in a backup and display the diff in the web interface.

The client should be able to:

1. Be on the Level 2 verify page
2. Click on a file that has changed
3. See the changes in the newest file

[hamstar]

Conventions for verification

The following conventions must be followed for backup storage during verification.

Working directory

Each archive verification above level 1 should have a working directory in /home/test/backup/verify/VERIFY_HASH/ where VERIFY_HASH is a unique hash made up of the date, and the two filenames.

• PHP method

function get_verify_hash( $filename1, $filename2 ) {
        return md5( $filename . mktime() . $filename2 );
}

• Ruby method

require 'digest/md5'
def get_verify_hash(filename1, filename2)
      Digest::MD5.hexdigest( "#{filename1}#{time.now}#{filename2}"
end

Archives will be stored in this working directory under their normal filenames and the prefixed with the source of the backup e.g. dropbox.2012.12.12.12.12.12 or local.2012.12.12.12.12.12. Then any files pertaining to the verification (such as diffs or sumlists) will be kept in the working directory.