feat: On chain content-addressable system

[mudler]

This card depends on https://github.com/masa-finance/masa-oracle/issues/380 .

As we have a on-chain ledger, now we want to be able to store large portion of data that can't fit in the ledger itself.

We want to keep the ledger state as lightweight as possible and we do not want to store large blobs of data in it, ideally because:

• these blocks should fit partly in memory
• we don't want all the node to have to store a chain that potentially could grow
• we want a solution that scales not depending on the size of the data that we want to store
• and we also don't want to make the updates to the ledger "slow".

Is your feature request related to a problem? Please describe. With a ledger, now we need to store permanent data somewhere. This card is about implementing a mechanism, similar to what is defined in https://github.com/masa-finance/masa-oracle/issues/345#issuecomment-2167399111 that allows to synchronize data across nodes in a distributed fashion, be able to push content and retrieve it via CID.

In this way the on-chain ledger can be just used as an index, while we store the data by using a specific protocol in the oracle network.

Describe the solution you'd like

• use something among the lines of bitswap https://github.com/ipfs/boxo/tree/main/bitswap to offload the blockchain content . for instance, CIDs are shared as gossip messages with part of the blockchain metadata, but the real content is delegated to bitswap.
• use raft to co-ordianate writing to the on chain ledger and the bitswap layer. https://github.com/libp2p/go-libp2p-raft/tree/master can be used for the purpose
• level-db can be used as well: https://github.com/ipfs/go-ds-leveldb

Describe alternatives you've considered using IPFS as backing storage, and have masa-oracles nodes be pinning services for the CID in the blockchain

Additional context

345

Blocked by: #380

Acceptance criteria

• [ ] we have a protocol in masa-oracle that allows nodes to synchronize persistent data across nodes
• [ ] only trusted nodes can write to the CID store
• [ ] the ledger references at least one CID store that is the content associated with our blockchain

[mudler]

I know that @jdutchak you want to work so hard on it ;) but let's assign the cards only when in progress