[Performance] Investigate and decrease RAM usage - add pebble kvstore - Githubissues

pokt-network / smt

A Go library that implements a Sparse Merkle Trie for a key-value map.

https://pkg.go.dev/github.com/pokt-network/smt

Other

16 stars 4 forks source link

[Performance] Investigate and decrease RAM usage - add pebble kvstore #52

Closed okdas closed 3 weeks ago

okdas commented 1 month ago

Summary

We experience a high memory usage on RelayMiners, and it appears the consumption is primarily driven by badger.

Human Summary

AI Summary

reviewpad:summary

Issue

https://github.com/pokt-network/poktroll/issues/551

Type of change

Please mark the relevant option(s):

[ ] New feature, functionality or library
[ ] Bug fix
[ ] Code health or cleanup
[ ] Documentation
[ ] Other (specify)

Testing

[ ] Run all unit tests: make test_all
[ ] Run all/relevant benchmarks (if optimising): make benchmark_{all | suite name}

Required Checklist

[ ] I have tested my changes using the available tooling
[ ] I have performed a self-review of my own code
[ ] I have commented my code (godoc format comments see: tip.golang.org/doc/comment)

If Applicable Checklist

[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have updated any relevant README(s)/documentation and left TODOs throughout the codebase
[ ] Add or update any relevant or supporting mermaid diagrams

okdas commented 1 month ago

I've made some changes to the BadgerDB while still working on it, and then also added Pebble. I think we're going to stick with Pebble for now as it has a lower memory footprint. We could go back later, or even better—make the store and its options configurable.

The poktroll PR is coming too.