search

gnolang / gno

Gno: An interpreted, stack-based Go virtual machine to build succinct and composable apps + Gno.land: a blockchain for timeless code and fair open-source
https://gno.land/
Other
842 stars 343 forks source link

gnovm store: `SetCachePackage` stores packages on memory. Possible OOM errors. #812

Open ajnavarro opened 1 year ago

ajnavarro commented 1 year ago

Description

After trying to reduce the memory usage moving from maps to an LRU cache here (https://github.com/gnolang/gno/pull/780), I realized that packages set on cacheObjects are not expected to be evicted, causing problems if we use an LRU cache.

Possible Solution

We can store these packages on persistent storage using a special key like /tmp/[UUID]/oid:[OID] where [UUID] is a unique ID created per storage session (every time you initialise a store or purge it)

When purging cache calling ClearObjectCache we generate a new UUID and we remove all keys under the previous UUID /tmp/[UUID]

On every server restart, we remove all keys under /tmp

WDYT?

thehowl commented 1 year ago

So your proposal is to have a LRU cache which additionally has some disk-backed data for cache misses?

Could another solution not be of requiring "temporary" / "throwaway" packages added with SetCachePackage to be finalised?

ajnavarro commented 1 year ago

@thehowl Yes, that might be another solution. However, we will need more research about the actual codebase to check where this method is used to ensure we can modify the behavior.