Document Mina daemon's databases

[georgeee]

Produce a document describing what are databases that Daemon uses.

Result of this task is expected to be a table of databases and descriptions.

Table's rows, I imagine, would be:

• Location of database
• Description
• Ocaml's type for key
• Ocaml's type for value

Relates to #13971

[georgeee]

This document is needed to analyze usage of RocksDB in block production/processing.

Major focus should be on frontier and ledger-related DBs.

[georgeee]

AFAIU all DBs we have are key-value storages.

And we probably have more than one key-value space within some of these DBs, let's have one "key-value space" per row in a resuting table.

To start with the task I suggest one launching a Mina node that connects to mainnet. And then checking what you have in .mina-config. Ideally we'd be able to inspect every DB with manual tool and check that there are no keys unaccounted, but it might be too tedious, so maybe just invetigating usages in codebase would be alright.

What I see on a mainnet's node:

• genesis/<...> <- three genesis ledger folders
• frontier <- transition frontier, where the recent blocks are kept
• wallets <- for locally stored secret keys, empty if no key configured
• mina_net2/<..> <- some databases used by networking implementation
  • mina_net2/block_db <- lmdb database for blocks, to be used in future by the new catchup algorithm
• root <- some data on the "root" transition, including uncommitted snarked ledgers
• trust <- database to store information about peers, now it's effectively containing peers we know and whether a peer is banned until some datetime

[Geometer1729]

root

The root/snarked_ledger location is defined here: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/transition_frontier/persistent_root/persistent_root.ml#L40 This path is given to Ledger.Db where Ledger is Mina_ledger.Ledger. So the snarked_ledger Db type gets defined here: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/mina_ledger/ledger.ml#L108-L120 which calls this functor: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/merkle_ledger/database.ml#L1

On this module: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/mina_ledger/ledger.ml#L89-L106

The Kvbd.t type it takes seems to be a generic database type with Bigstring.t for both key and value. But from calls to Kvdb like this: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/merkle_ledger/database.ml#L101-L102 It looks like the keys represent serialized Location.ts which I think is this enum: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/merkle_ledger/location.ml#L54-L55

and the value appears to be the Toeken_id.Set.t type from the inputs. I'm not sure where Token_id comes from, there are a few opens at the top of the file.

The root/root location is defined here and seems to store a hash of the genesis state.

[Geometer1729]

trust

src/lib/trust_system/peer_trust.ml

Has the key as Peer_id.t which I believe refers to this: https://github.com/MinaProtocol/mina/blob/10a0bf9b8b5c27407b349f02ecaeea964e14690e/src/lib/trust_system/peer_trust.ml#L252-L256

and the value Record.t which I beleive reffers to this https://github.com/MinaProtocol/mina/blob/10a0bf9b8b5c27407b349f02ecaeea964e14690e/src/lib/trust_system/record.ml#L5-L12

[Geometer1729]

frontier

It looks like this module: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/transition_frontier/persistent_frontier/database.ml#L212

calls this functor: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/rocksdb/serializable.ml#L53

With this argument: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/transition_frontier/persistent_frontier/database.ml#L29

I think that results in the key type being this private Enum: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/transition_frontier/persistent_frontier/database.ml#L49-L56

Here Root_data.Minimal.Stable.V2 refers to this type: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/transition_frontier/frontier_base/root_data.ml#L114

and I think this defines a type family like thing for what the value types is depending on the key: https://github.com/MinaProtocol/mina/blob/4495af5caea5e1bb2f98f92592c065f93a586ade/src/lib/transition_frontier/persistent_frontier/database.ml#L74-L86