search

gaffneyc / heroku-buildpack-jemalloc

Heroku buildpack that installs the Jemalloc memory allocator
BSD 3-Clause "New" or "Revised" License
248 stars 75 forks source link

Purge old jemalloc files from cache on cache miss (eg on stack or jemalloc version change) #37

Open edmorley opened 3 months ago

edmorley commented 3 months ago

This buildpack compiles jemalloc during the build and caches the result under a stack and jemalloc-specific path: https://github.com/gaffneyc/heroku-buildpack-jemalloc/blob/a7fe9271f64c861debe9b6cc814203c17e7e68dc/bin/compile#L37-L38

This correctly ensures that a recompile happens if the stack or requested jemalloc version changes.

However, currently the buildpack does not remove these cached items on cache miss, meaning the cache will continue to grow in size each time the stack or requested jemalloc version changes (unless users know to manually clear their build cache using a special CLI plugin).

To resolve this, the buildpack could do an rm -rf "${CACHE_DIR}/jemalloc/" on cache miss before the download/recompile, here: https://github.com/gaffneyc/heroku-buildpack-jemalloc/blob/a7fe9271f64c861debe9b6cc814203c17e7e68dc/bin/compile#L116-L121

gaffneyc commented 3 months ago

My original intent with not clearing the cached builds was to make it possible to revert to a previously built version of jemalloc without having to deploy. Realistically though that's confusing since you have to build a new slug in order to use a version you've never deployed before.

Purging the builds would probably work but I'd like to noodle on it. Fortunately each build is ~6MB and the cadence of Heroku stack and jemalloc releases is both rather slow.

Another option would be to only clear the cache for stacks other than the current one. Compressing the build artifacts would also reduce them down to ~2MB.

edmorley commented 3 months ago

Ah if it's only 6MB that's not so bad - I had imagined it would be larger than that.