Tiered DataStore Implementation

[justin0mcateer]

The documentation for this module discusses 'Tiered' datastores and there is an implementation in the original Python module. However, I don't find anything in the documentation, source code or in the discussions relative to any Tiered implementation or configuration.

Is this implemented somewhere (perhaps implicitly) and I am just missing it, or has it never been implemented? If not, is this something that would be at all of interest to the broader community?

[welcome[bot]]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review. In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment. Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[hsanjuan]

Tiered is mentioned in the context of caching etc. In practice, because the datastore was mostly using ipld blocks, these optimizations were introduced at the Blockstore layer (https://github.com/ipfs/boxo/tree/main/blockstore). In practice, as well, the good datastore implementations already provide tiered access (Badger, Pebble have in-memory caches and even flatfs benefits from OS-provided caching. There are cases when all the caching layers involved are actually causing negative performance impact.

[aschmahmann]

At the moment there doesn't seem to be much of a need for it. However, if someone wants to use something like that in their application they can either:

1. Write their datastore elsewhere and have it conform to the same interface as here
2. Make a PR adding the datastore here

If someone wants to use this it might be better to start with it embedded with the relevant application and then made as a PR here if it seems generally useful. Closing for now though.