Migrate away from sled

[elsirion]

Sled seems not mature enough and development has slowed down. In the beginning I thought they might get to a stable release before MiniMint, but that's probably too optimistic.

I'd like to stick to dumb key-value stores with transactions, prefix-fetching and deterministic ordering because there seems to be less room for accidental consensus-incompatibility that way (e.g. complicated SQLite queries would be hard to analyze for being deterministic on all platforms). I'd also like the database to be embedded to reduce complexity (currently, if there's a DB error it's safe to assume something went horribly wrong, like the underlying drive unmounting, and we can shut down). That leaves a few options off the top of my head:

• LevelDB: my current favorite, used in Bitcoin Core, so already a dependency
• SQLite as a simple KV store: probably also a great choice, feels hacky though
• Berkeley DB: used in Bitcoin Core, but has caused problems historically afaik

I did not deeply evaluate any of these options and would be happy to hear opinions on these or other KV stores.

EDIT 1: I confused LMDB with LevelDB, fixed

[DON-MAC-256]

Do we have a tracking issue on the Sled repo for stabilization?

Whatever the database backing the system, it should be able to be swapped quickly via proper API wrappers.

[elsirion]

Do we have a tracking issue on the Sled repo for stabilization?

Idk if that's still realistic? The README contains some future goals at the end that say "1.0 imminent", but there seem to be severe bugs open and little actual development happening (latest change was copyright stuff, before that 3m ago).

Whatever the database backing the system, it should be able to be swapped quickly via proper API wrappers.

That's fortunately already the case :smiley: it's not well documented, but the main trait is pretty straight forward (the raw version is for enabling trait objects, the normal version is for direct use). There's already a in-memory impl backed by a BTreeMap.

[Kixunil]

Rocksdb also exists and has Rust bindings. electrs uses it for instance. AFAIU it's a fork of LevelDB but the difference looks marginal to me.

[maan2003]

I vote for Rocksdb.

• Rust bindings look well maintained
• straight forward way to implement prefix interator

[elsirion]

I'm open to any DB as long as it's embedded. External ones would introduce just another point of failure.

One thing to keep in mind: ordering of retrieved items. I kinda assume that any DB that supports prefix iteration has a stable order of entries. I don't think it's currently consensus critical but I generally tried to get away from Hash* data structures because they can give you a very hard time debugging why something doesn't work due to different ordering when iterating.

[ahv15]

Is anybody working on this issue at the moment? If not I'd like to give it a try.

[elsirion]

Is anybody working on this issue at the moment?

Not me at least, would be great!

[elsirion]

@ahv15 any progress? Would be interesting to #50. Depending on if I can get proper ACID transactions I could remove the conflict filter which has been annoying me for quite a while.

[elsirion]

I'm growing very fond of the SQLite idea by now :grimacing: seems like neither the RocksDB nor the LevelDB rust bindings expose the transaction functionality, making them unsuitable for the job and I don't fancy writing my own C binding.

[Mo0nbase]

Taking a look at this issue we certainly need to move away from sled unfortunately there is basically no native integrated DB solution which isn't either in beta and missing critical features or would require ugly/hacky code to work. People above have suggested using RocksDB for the database for which there is a great up and coming native rust port here. It exposes nearly the entire feature set but as mentioned above it is lacking TransactionDB support but the issue has been brought up before in 2019 and several code revisions leading to this issue. Although it hasn't been merged into main it looks nearly complete so RocksDB is a possibility we should consider (activity on the issue should also push it forward). I'll also mention that it is one of the few databases being actively maintained and has a stable development history for rust. The only other viable alternative which @elsirion has made clear is not desirable would be a complete refactoring to a SQL based external system using a stable and feature complete repository like tokio-postgres or as mentioned above a K, V solution with SQLite. These both could introduce several points of failure not to mention the fact that RocksDB performance, feature set, and native stability will make mini-mint much more robust and powerful as the rust community expands and all projects involved grow bigger. I might be wrong about parts of this but I'm of the mind we should create a testing branch or at least a pr where we can start development on a RocksDB solution using the suggested repository and the TransactionDB pr while we wait for it to be merged into main.

P.S. LevelDB would also be a great solution however it has even less rust developers working on a port and all repositories are not actively maintained

[elsirion]

If that RocksDB PR gets merged it would probably be my top choice (still have to look at their transaction API, but I assume it to be usable). @casey (I think) also pointed me to redb, which looks interesting but is much less mature.

Do you want to take lead on this @Mo0nbase? The migration would probably happen in 2 steps:

1. Implement the current Database trait for RocksDB. That should be rather straight forward.
2. Change the architecture such that we get rid of the home-grown DbBatch "transaction" mechanism and use actual transactions. #50 laid the foundation for that, I'm happy to help with the API changes (assuming you are new to rust from your GH profile).

[justinmoon]

Probably a longshot, but it seems like the creator of sled is working on some kind of new database https://github.com/komora-io / https://twitter.com/komora_io/status/1522192451485810688.

Anyone made any progress here?

[justinmoon]

Interesting post about using sqlite as k:v store https://github.com/the-lean-crate/criner/issues/1

[elsirion]

You still on this @Mo0nbase?

[Mo0nbase]

Yes a PR for this is right around the corner.

[elsirion]

Feel free to open it as a draft PR even before you think it's ready ;)

[justinmoon]

Closed by #173