Decide upon data storage

[oubiwann]

Sled?

[oubiwann]

https://persy.rs/ ?

[oubiwann]

std:HashMap?

We could:

1. encrypt the entire hashmap: https://docs.rs/enc_file/latest/enc_file/fn.encrypt_hashmap.html
2. decrypt it with a main pass in-mem (as-needed) using secrecy
3. only the keys would be plain-text at that point, the values encrypted with the main pass

Would probably be a pain managing the writes ... could just do the same with persy or sled.

[oubiwann]

I think I'll just start with Sled and see what issues I run into ...

[oubiwann]

Another possibility: https://crates.io/crates/evmap

[oubiwann]

SurrealDB?

• https://docs.rs/surrealdb/1.0.0-beta.8/surrealdb/struct.Datastore.html
• https://github.com/surrealdb/surrealdb

[oubiwann]

Surreal offers file-based encryption -- gonna go with this for now.

[oubiwann]

SurrealDB is a MASSIVE dependency and it's slowing down all the testing by orders of magnitude.

So, re-opening ...

[oubiwann]

What about going SUPER simple?

Store everything in-mem, in a hashmap ...

Startup:

• Read the file to bytes and decrypt
• De-serialise (https://crates.io/crates/bincode)
• Set up concurrent access

Run-time (both CLI and a daemon):

• Access the data as an in-mem hashmap
• Regularly backup to disk (serialise, encrypt, write)

Shutdown:

• Lock access
• Serialise in-memory
• Encrypt in-memory
• Write to disk

Seems like age would be a good fit here ...

[oubiwann]

Going to experiment with this next:

• https://github.com/paritytech/trie
• https://docs.rs/memory-db/0.31.0/memory_db/struct.MemoryDB.html

[oubiwann]

Now this:

• https://github.com/rust-lang/hashbrown

[oubiwann]

Now this:

• https://docs.rs/dashmap/5.4.0/dashmap/struct.DashMap.html

[oubiwann]

Made more progress with that last one than anything else, to date ... gonna try closing this again.