micromamba is not statically linked as documentation claims

oursland commented 2 months ago

Troubleshooting docs

[X] My problem is not solved in the Troubleshooting docs

Anaconda default channels

[X] I do NOT use the Anaconda default channels (pkgs/* etc.)

How did you install Mamba?

Micromamba

Search tried in issue tracker

static

Latest version of Mamba

[X] My problem is not solved with the latest version

Tried in Conda?

Not applicable

Describe your issue

According to the user documentation, micromamba is claimed to be a "fully statically-linked, self-contained, executable", however this is not the case.

On macOS, installed using brew install micromamba here's the output of otool -L:

otool -L /opt/homebrew/opt/micromamba/bin/micromamba
/opt/homebrew/opt/micromamba/bin/micromamba:
    @rpath/libmamba.2.dylib (compatibility version 2.0.0, current version 2.0.0)
    /opt/homebrew/opt/libsolv/lib/libsolv.1.dylib (compatibility version 1.0.0, current version 0.0.0)
    /opt/homebrew/opt/libsolv/lib/libsolvext.1.dylib (compatibility version 1.0.0, current version 0.0.0)
    /usr/lib/libarchive.2.dylib (compatibility version 9.0.0, current version 9.2.0)
    /opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.5)
    /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 9.0.0)
    /opt/homebrew/opt/openssl@3/lib/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
    /opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
    /usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.8)
    /opt/homebrew/opt/yaml-cpp/lib/libyaml-cpp.0.8.dylib (compatibility version 0.8.0, current version 0.8.0)
    /opt/homebrew/opt/reproc/lib/libreproc++.14.dylib (compatibility version 14.0.0, current version 14.2.4)
    /opt/homebrew/opt/reproc/lib/libreproc.14.dylib (compatibility version 14.0.0, current version 14.2.4)
    /opt/homebrew/opt/fmt/lib/libfmt.10.dylib (compatibility version 10.0.0, current version 10.2.1)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1700.255.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)

While macOS requires the use of dynamic linking for some systems-libraries (i.e. libSystem.B.dylib) due to the lack of a stable ABI suitable for static linking, it seems that libraries such as libzstd, libopenssl, etc should be statically linked in.

mamba info / micromamba info

No response

Logs

No response

environment.yml

No response

~/.condarc

No response

ickc commented 3 weeks ago

That's because you're using the one provided by homebrew. See how they compile it: https://github.com/Homebrew/homebrew-core/blob/master/Formula/m/micromamba.rb#L58

Secondly, there's no true static linking on macOS. If you use the binary provided officially here, you will still see some:

% otool -L $(which micromamba)
~/.local/bin/micromamba:
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.106.0)
        /System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 59754.41.1)
        /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos (compatibility version 5.0.0, current version 6.0.0)
        /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1109.40.9)
        /usr/lib/libc++abi.dylib (compatibility version 1.0.0, current version 904.4.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)

adam-azarchs commented 2 weeks ago

The linux binaries aren't statically linked either, either the outputs from https://github.com/mamba-org/mamba/blob/main/.github/workflows/static_build.yml or the downloads from conda-forge:

$ readelf --needed-libs micromamba
NeededLibraries [
  ld-linux-x86-64.so.2
  libc.so.6
  libdl.so.2
  libm.so.6
  libpthread.so.0
  libresolv.so.2
  librt.so.1
]

There are legitimate use cases for an actually-static build, e.g. as part of a multi-stage docker build where one might want to generate an environment using a minimal container that doesn't have glibc (e.g. Alpine) for use in later stages.

I do understand not wanting to support such use cases, but then please don't claim that the binary is "a fully statically-linked, self-contained, executable".

oursland commented 2 weeks ago

The issue is particularly concerning for package managers on Windows. In Windows you cannot simply replace a linked library when performing an update, as you can under Linux and macOS.

In Linux and macOS, the unlink() will only result in removing the named file, but the actual file deletion will take place when all open file handles are closed. This ensures that running binaries can continue to execute as expected.

On Windows, the deletion of a file causes it to be deleted immediately and the memory mapped portions within running executables to be made invalid. Consequently when a package manager (e.g. mamba and micromamba) upgrade a dependency, the executable may crash if the dependency's code is executed. This is particularly present with zstd, which is upgraded then attempted to use when decompressing the next package file.

Package managers on Windows may work around this by first renaming the dependent libraries, performing the upgrade, then deleting the renamed files at exit.

Alternatively, the package manager can be statically linked and obviate the need for this workaround. The choice to move to a dynamically linked micromamba, despite the documentation claiming to be a statically linked binary, has very serious consequences that package managers must consider.

mamba-org / mamba