search

valkey-io / valkey

A flexible distributed key-value datastore that supports both caching and beyond caching workloads.
https://valkey.io
Other
15.49k stars 570 forks source link

[NEW] Consider support for mimalloc allocator #346

Open jirutka opened 4 months ago

jirutka commented 4 months ago

Can you please consider adding support for mimalloc (v2) as an alternative to jemalloc? mimalloc is faster in benchmarks and smaller than jemalloc.

Related issues:

madolson commented 4 months ago

I have nothing against this, especially if it's done in a way that users can opt-in to. We do have some special code to allow for active defragmentation of Jemalloc arenas, so we might also want to figure out how to implement that in mimalloc. Ideally the next step would be for someone to put together a PoC and validate the performance is actually better.

zuiderkwast commented 4 months ago

Agreed. Contributions are welcome.

PingXie commented 4 months ago

We do have some special code to allow for active defragmentation of Jemalloc arenas, so we might also want to figure out how to implement that in mimalloc.

defrag is enabled explicitly only when jemalloc is used so this should be fine. in other words, you get defrag with jemalloc only.

I am aligned with adding mimalloc as long as we don't vendor it. On a related note, I think we should move forward and remove the one-off je_get_defrag_hint patch and instead use more generic heuristics for defrag so that other allocators could benefit from the defrag feature as well. jemalloc should be devendored too.

yairgott commented 4 months ago

There are indications that jemalloc is faster and uses less memory than mimalloc. See for example: https://github.com/rust-lang/rust-analyzer/issues/1441#issuecomment-2016419122

The used jemalloc version is quite old. The main challenge is related to porting this defrag patch. We need to check what are the alternatives.