Support versioned database formats

[oubiwann]

This works comes on the heels of several efforts:

• 54 (won't do)

• 59 (requires examining db files at the byte-level)

• 48

• 44

All of these have led to me working on the following tasks:

• [x] Tag our database files with version metadata
  • [x] create a wrapper struct for bytes and the current version
  • [x] refactor DB write code
  • [x] update the DB write to add an extra step of including the version metadata in the file
  • [x] refactor the DB read code
  • [x] update the DB read to properly parse the version metadata
• [x] Add CLI commands for working with database file/format versions
• [x] Create example databases in older formats and save them for use in tests
• [x] Add logic for parsing the version, and on error, attempting to open DBs in the old format
• [x] #61

[oubiwann]

Interestingly ... when running locally, I see success for 0.5.0 (after an initial error):

>> Read an old database (v0.5.0)
2023-01-23 02:42:05 ERROR [src/store/db/versioned.rs:21 rucksack::store::db::versioned] ▶ couldn't deserialise versioned database file: Utf8 { inner: Utf8Error { valid_up_to: 52, error_len: Some(1) } }

URL                                      | User / Account                 | Access Count
-----------------------------------------+--------------------------------+-------------
http://example.com                       | clammy                         |      0      

but failure for 0.6.0:

>> Read an old database (v0.6.0)
2023-01-23 02:42:05 ERROR [src/store/db/versioned.rs:21 rucksack::store::db::versioned] ▶ couldn't deserialise versioned database file: Utf8 { inner: Utf8Error { valid_up_to: 52, error_len: Some(1) } }
2023-01-23 02:42:05 INFO [src/store/db/mod.rs:79 rucksack::store::db] ▶ Given database appears to be non-versioned; attempting old format ...
2023-01-23 02:42:05 INFO [src/store/db/mod.rs:110 rucksack::store::db] ▶ couldn't deserialise bincoded hashmap bytes: Utf8 { inner: Utf8Error { valid_up_to: 39, error_len: Some(1) } }
2023-01-23 02:42:05 INFO [src/store/records/v070.rs:32 rucksack::store::records::v070] ▶ couldn't deserialise bincoded hashmap bytes: Utf8 { inner: Utf8Error { valid_up_to: 39, error_len: Some(1) } }
2023-01-23 02:42:05 INFO [src/store/records/v070.rs:33 rucksack::store::records::v070] ▶ Attempting to decode hashmap from previous version (0.6.0)
2023-01-23 02:42:05 INFO [src/store/records/v060.rs:30 rucksack::store::records::v060] ▶ couldn't deserialise bincoded hashmap bytes: OtherString("invalid value: integer `2`, expected variant index 0 <= i < 1")
2023-01-23 02:42:05 INFO [src/store/records/v060.rs:31 rucksack::store::records::v060] ▶ Attempting to decode hashmap from previous version (0.5.0)
2023-01-23 02:42:05 ERROR [src/store/records/v050.rs:17 rucksack::store::records::v050] ▶ couldn't deserialise bincoded hashmap bytes: OtherString("invalid value: integer `2`, expected variant index 0 <= i < 1")

(I suspect the error with 0.6.0 may go away after #61 is merged.)

Contrastingly, on CI/CD, both tests for older versions of the old DB err out:

>> Read an old database (v0.5.0)
2023-01-23 08:46:45 ERROR [src/store/db/encrypted.rs:53 rucksack::store::db::encrypted] ▶ could not decrypt data: aead::Error
>> Read an old database (v0.6.0)
2023-01-23 08:46:45 ERROR [src/store/db/encrypted.rs:53 rucksack::store::db::encrypted] ▶ could not decrypt data: aead::Error

Not sure why decryption is failing on CI/CD ... different architecture. Endianness? Dunno ... maybe some arch-independent encoding would help?

[oubiwann]

As for the endianness:

• v0.5.0, v0.6.0, and v0.7.0 all use bincode 2.0.0-rc2
• 2.0.0-rc2 uses little endian by default, with not changes based upon architecture

[oubiwann]

Versions are being properly parsed, etc. However, I'm seeing an odd issue where at least one command (or one series of operations) results in the DB not being able to be decrypted.

Oh, this has a ticket: #59