heroku / base-images

Recipes for building the base images for Heroku's stacks
BSD 3-Clause "New" or "Revised" License
267 stars 78 forks source link

Heroku-24: Add `binutils` to the run image #306

Closed edmorley closed 6 months ago

edmorley commented 6 months ago

Python's ctypes.util.find_library is used to find libraries at runtime. find_library() calls out to various external programs to help with this task, initially trying ldconfig, and then falling back to gcc and ld if needed - as documented here: https://docs.python.org/3/library/ctypes.html#finding-shared-libraries

Even after the removal of compilation tools from the run image in #273 (which removed both ld and gcc), ldconfig still exists in the run image, so libraries in the base image can still successfully be found by ctypes.util.find_library.

However, whilst the find_library() docs say that it supports LD_LIBRARY_PATH, the docs fail to mention that the env var is only honoured when ld can be found: https://github.com/python/cpython/blob/e870c852c0ea96fa4e4569e9c39c7ceb80ce858d/Lib/ctypes/util.py#L315-L320

As such, when only ldconfig is available (and not ld), find_library() only returns libraries from the base image, and not any installed via eg the APT or other buildpacks (which rely upon LD_LIBRARY_PATH since they have to do userland "installs" of libraries due to not having root permissions).

One such popular use of ctypes.util.find_library is in Django's GIS feature, which uses it to locate the GDAL library at runtime - which will stop working with buildpack-installed GDAL if ld is not in the run image: https://github.com/django/django/blob/4971a9afe5642569f3dcfcd3972ebb39e88dd457/django/contrib/gis/gdal/libgdal.py#L53-L66

Therefore, we sadly have to include binutils (which is the package that provides ld) in the run image after all - though thankfully doing so only forgoes 19 MB of the original 203 MB run image size reduction from removing gcc, make and libc6-dev - so we're still well ahead.

GUS-W-15821115.