MichaelChirico
commented
4 years ago I can confirm this issue.
My use case: I develop both a large webapp using the R-package shiny as well as a RESTful API service using the R-package plumber. Both services deal with large amounts of data (up to 10GB) and both services are replicated multiple times. As malloc is not returning freed memory, this causes a memory leak in the order of 100 GB for me.
My solution: As Dmitriy Selivanov proposed, I switched to jemalloc which fixed all problems and doesn't seem to have any unwanted side effects. I'm running this for half a year now.
Best wishes, Lukas
METADATA
- Comment author - Lukas Klass
- Timestamp - 2019-12-12 08:40:51 UTC
Opening this "WISHLIST" issue ticket as was suggested by Luke Tierney in email about year ago (sorry for that).
Issue is that in systems which use glibc for memory allocation malloc doesn't return memory to the system. See discussion here - https://stat.ethz.ch/pipermail/r-devel/2017-August/074761.html
About year ago I've created a repo for experiments - https://github.com/dselivanov/r-malloc. Seems jemalloc, tcmalloc and Simon's mallinfo::malloc.trim(0L) successfully release memory to OS.
On my side I can add that I've switched to jemalloc in all projects where I use R as long-running backend service. Haven't notice any issues.
Initial context is described in r-devel thread:
METADATA