search

ysbaddaden / gc

A garbage collector for Crystal
93 stars 5 forks source link

Resizeable memory maps #17

Open ysbaddaden opened 2 years ago

ysbaddaden commented 2 years ago

GC_map() currently tells the kernel to allocate as much (virtual) memory as possible, but we could be smarter about it. Instead of asking for as much memory as there are on the computer (thrice), we could tell to map as much as we need, and grow the map as we grow the GC HEAP using mremap(2).

Ideally, we could consider to shrink it after a collect (or once every collect) if we could release a significant amount of memory back to the OS.

Note: allocating so much memory causes issues with Valgrind that intercepts the mmap call and injects the MAP_FIXED flag which tells the kernel to fully allocate memory, which isn't possible and crashes the program (oops).

ysbaddaden commented 2 years ago

I hit some issues while trying to implement this:

munmap can shrink a previously allocated mmap, but this is definitive: the memory is no longer accessible (trying to access it will segfault). Despite being capable to pass an address, we can't mmap the released memory again to "grow" the map again. The address is a mere hint and the memory is allocated anywhere (at least on Linux).

mremap should be capable to shrink/grow a mmap, but it's only available in Linux and hidden behind the _GNU_SOURCE flag. It's not portable to anything else (e.g. Darwin, FreeBSD or OpenBSD).

An alternative could be to use sbrk to dynamically grow or shrink the DATA section (beware to copy the original end of the DATA section for the collector to not walk the GC HEAP :fearful:). It's legacy but should be broadly available.

ysbaddaden commented 2 years ago

Except that we need to maps: one for the small objects and another one for the large objects, while sbrk only gives us access to a single one :facepalm: