search

google / tcmalloc

Apache License 2.0
4.31k stars 463 forks source link

tcmalloc is trying to keep everything allocated for longer #215

Open coder-saab001 opened 9 months ago

coder-saab001 commented 9 months ago

I am working on an application which is reading continuously from network and let's say storing it somewhere. I am reading in say 1 MB chunks again and again. I want the tcmalloc memory of previous iteration to be freed before next read iteration. What I can see now is it takes about ~1 sec for tcmalloc to free the previously allocated memory. Is there any way to tweak this default behaviour of tcmalloc?

ckennelly commented 9 months ago

You can call MallocExtension::ReleaseMemoryToSystem to speed up the process, but there's some smoothing based on past memory usage.

If another 1MB allocation is going to likely occur, it's faster to keep it cached than to return it to the OS (a syscall) and then reuse it (page faults).

coder-saab001 commented 9 months ago

We want to minimise the memory usage. Say currently 16 chunks of 1 MB are in memory. In next iteration, next 16 chunks are about to come. We want to get that first 16 MB got release before second 16MB is fed into the memory. Is that possible somehow?

junyer commented 8 months ago

I doubt that any answer given here will differ substantially from the answer given on https://github.com/gperftools/gperftools/issues/1465.