search

microsoft / mimalloc

mimalloc is a compact general purpose allocator with excellent performance.
MIT License
9.74k stars 793 forks source link

Add an Emscripten/wasm implementation of primitives #822

Closed kripken closed 4 months ago

kripken commented 8 months ago

This PR adds a prim implementation for the Emscripten wasm toolchain. There is already a WASI wasm prim in mimalloc, which this is separate from. WASI tends to be used on the server, and Emscripten on the browser, and there are API differences that led to a different design here. Another difference is that Emscripten has stable support for multithreading, and so that part is fully implemented here (it was the main reason for the port, actually - to get a malloc that scales well with multiple cores).

The overall idea in this implementation is to build on top of emmalloc as the "system allocator". emmalloc is a minimal allocator in Emscripten, so it can be used to provide a VirtualAlloc-like API. This avoids using sbrk directly, which could lead to problems with returning memory to the OS (fragmentation and leaks).

There is a PR up on the Emscripten repo with all the integration into our toolchain so that people can test it out there. Also some charts with numbers, which look great!

Thank you for the mimalloc project :heart: it was very easy to build support for, and as the numbers there show now we have an option for a malloc implementation that looks like it scales really well, in fact just as good as native builds.

kripken commented 7 months ago

@microsoft-github-policy-service agree [company="google"]

kripken commented 7 months ago

@microsoft-github-policy-service agree company="google"

daanx commented 4 months ago

Ah this is awesome -- apologies for the later reaction! I love emscripten -- the Koka compiler uses it as well and it would be good to have a more native implementation.

daanx commented 4 months ago

I merged your PR manually into the dev branch (as you based on master) -- hope that is ok with you. I added credits of course. This looks great -- and good to hear the performance seems good. The arena reserve should be 256MiB at a time so I think the alignment on 4MiB is not a real problem. Within the large arena's mimalloc will efficiently (re)use the memory (so even using sbrk will not be a problem as mimalloc will manage large chunks at a time.

kripken commented 4 months ago

Great, thanks for merging! And sorry I didn't know the PR should target dev.

I'll close this PR then.

Btw, recently I saw a 5x speedup using mimalloc over dlmalloc and we have other positive reports coming in from other Emscripten users. Thanks again for mimalloc!