search

johan-bolmsjo / jbmalloc

Scalable memory allocator implementing malloc(3) with friends.
0 stars 0 forks source link

[Bug] Compile problem #1

Open zuiderkwast opened 3 years ago

zuiderkwast commented 3 years ago

When I try to build it, it get make: *** No targets specified and no makefile found. Stop.

Maybe it's because there is no Makefile and also the source code seems to be hidden in invisible files (for security reasons???). Please help! I urgently need a decent allocator.

johan-bolmsjo commented 3 years ago

That is correct. This repo is the reason I created my github account a long time ago. The memory allocator actually exist and is pretty good but I used it in a commercial product and then it became a legal gray zone to release it here. I did not want to risk it. I've kept the empty repo for sentimental reasons.

I suggest you look into one of the following memory allocators which are all high performance and probably better than my unreleased allocator in many respects:

zuiderkwast commented 3 years ago

Thanks for your informed wisdom! So the rumors that jbmalloc is pretty good is true then. Jemalloc is a bit slow for huge allocations. In particular, we need fast realloc for power of two sizes. Apparently, jemalloc was using mremap a long time ago but not anymore because Linux is using some linear scan which can become slow when there is kernel space fragmentation. Are you using mremap in user plane?

johan-bolmsjo commented 3 years ago

jbmalloc would unfortunately not be good for that use case either. Perhaps something for version 2 :-) It does not use mremap either and scales poorly with large allocations.

The main inspiration for jbmalloc was these three papers:

1) http://phk.freebsd.dk/pubs/malloc.pdf

Keeping meta data separate from allocated memory to reduce risk of heap corruption causing crashes in the allocator. Of course this leaves memory corruption crashes for application developers to sort out but I think it's a better trade off.

2) www.usenix.org/publications/library/proceedings/bos94/full_papers/bonwick.a (text file)

This is the basic slab allocator idea. I think it's used by most new allocators as it's easy to make it scale with multiple threads.

3) http://www.parrot.org/sites/www.parrot.org/files/vmem.pdf

Ideas on how to scale the basic slab allocator to multiple threads.

I've not looked into mimalloc much but both jemalloc and tcmalloc are pretty similar to my allocator (or the other way around). They are more advanced in implementing multi thread scaling for large allocations. I did not care at all about that problem and concluded that most allocations are small and only optimized for that case.