Start CacheKV store v2

[ValarDragon]

What is the purpose of the change

Start CacheKV store v2, that separates responsibilities, cleans up code significantly.

Should eliminate dead lock risks, and significantly improve performance.

TY to @dangush for helpful discussions around spec leading to this. (Feel free to help contribute with this!)

The architecture of this is based on separating the roles of whats going on.

We acknowledge that the cache kv store has two responsibilities, and explicitly make two different internal structs and compose them together. Responsibilities

• WriteBuffer
• ReadCache

This lets us re-use and improve each component independently, keeps each component pretty simple, and advance the clear API breaking roadmaps for improvement for each. (And joint performance measurement / improvement remains pretty simple)

Current status

Base write buffer written, need to add iterator.

Barely any docs

Brief Changelog

(for example:)

• The metadata is stored in the blob store on job creation time as a persistent artifact
• Deployments RPC transmits only the blob storage reference
• Daemons retrieve the RPC data from the blob cache

Testing and Verifying

(Please pick one of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added unit test that validates ...
• Added integration tests for end-to-end deployment with ...
• Extended integration test for ...
• Manually verified the change by ...

Documentation and Release Note

• Does this pull request introduce a new feature or user-facing behavior changes? (yes / no)
• Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? (yes / no)
• How is the feature or change documented? (not applicable / specification (x/<module>/spec/) / Osmosis docs repo / not documented)

[alexanderbez]

Super stoked for this @ValarDragon. LMK how I can help.

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.