Request: a Cache evicted by max allowed size

[beckend]

So the application doesn't run out of memory.

[wvwwvwwv]

Hi! Could you try HashCache? It limits the maximum size of the container thereby being able to help prevent OoM.

[beckend]

The problem is when a value is 1-10MB each, some are 1KB, so in this case the limit of keys are hard to use.

[wvwwvwwv]

Oh, interesting problem. I guess a new type of memory allocators that raises an error when a certain threshold is reached will be required, e.g., using the allocator_api implementing it will be possible. => if AllocError is raised, evict an entry from the container and retry allocation. However, not sure if this crate is the right place to implement it.

[beckend]

I am saying just evict the oldest items when user limit is reached, you already know the value size, evict as many you need until the threshold is reached. Let us say the user gives 1G limit, when cache insert is done, you start evict check based on this user value.

[beckend]

Also would be useful for the user to be able to update the size while instance has been created, so user can do dynamic resize if more ram is available.

[beckend]

Basically your don't handle any memory for the user, only allow them to set the total max accumulated key+value size before eviction.

[wvwwvwwv]

It’s not that trivial because the crate is oblivious of any indirect heap memory allocation incurred during construction of K and V; it knows the size of K and V as they are Sized but it does not account for any Box or Arc in K and V. Accounting for the allocated heap memory size is a memory allocator’s job - AllocError is one way a memory allocator can give information to the consumer - ex: this crate - and then the consumer will be able to react to it - ex: this crate evicts an entry.

So I’d suggest that you build a custom memory allocator using the allocator_api then this crate will be able to adopt it. What do you think of it?

[beckend]

Got it, thanks.