When using system packages from within a python application symbol lookup error may happen.
This happens when the system package (like git) is linked against a different version of libc compared to the python version (from devenv) is linked against.
A specific case where this happens more often occurs when using a remote git repository as a requirement for your devenv-based python project. devenv calls pip install -r requirements.txt, which will run git. git is usually not defined inside devenv, thus the required libc is likely to be different from what is used in devenv. The python wrapper within devenv runs python with LD_LIBRARY_PATH=$DEVENV_PROFILE/lib, which will force ld to use libraries within $DEVENV_PROFILE/lib. The libc that resides in $DEVENV_PROFILE/lib is a different version than what git assumes to be ld-ed against.
This results in errors like the following:
Collecting git+https://github.com/pypa/sampleproject.git (from -r /nix/store/g41hqm03k2179v5pkyv889gyrgb370g0-requirements.txt (line 1))
Cloning https://github.com/pypa/sampleproject.git to /tmp/nix-shell.07Ag0j/nix-shell.tKP4lT/pip-req-build-jidkerj5
error: subprocess-exited-with-error
× git version did not run successfully.
│ exit code: 127
╰─> [2 lines of output]
git: error while loading shared libraries: __vdso_gettimeofday: invalid mode for dlopen(): Invalid argument
[end of output]
Potential solutions
Adding git to packages in devenv will resolve the problem with git, but on my system I got the next binary that resulted in the a similar error (gh). Adding all personal tools is one way to get over this problem.
Avoid using other binaries within the python application by not using git.
Resetting LD_LIBRARY_PATH upon calling other binaries from within the wrapped devenv python.
Make sure Python libraries are always linked against specific (working) versions of native libraries. This can be done using patchelf/autopatchelf. In addition, LD_LIBRARY_PATH should not be used so that ld won't be forced to use different libraries. See PR https://github.com/cachix/devenv/pull/840 as a poc.
Also running into this when calling system commands from within my Python application. Current band-aid solution is to reset LD_LIBRARY_PATH before doing any system command.
Describe the bug
When using system packages from within a python application
symbol lookup error
may happen. This happens when the system package (likegit
) is linked against a different version oflibc
compared to the python version (from devenv) is linked against.A specific case where this happens more often occurs when using a remote git repository as a requirement for your devenv-based python project. devenv calls
pip install -r requirements.txt
, which will rungit
.git
is usually not defined inside devenv, thus the requiredlibc
is likely to be different from what is used in devenv. Thepython
wrapper within devenv runs python withLD_LIBRARY_PATH=$DEVENV_PROFILE/lib
, which will force ld to use libraries within$DEVENV_PROFILE/lib
. Thelibc
that resides in$DEVENV_PROFILE/lib
is a different version than whatgit
assumes to be ld-ed against.This results in errors like the following:
Potential solutions
git
topackages
in devenv will resolve the problem with git, but on my system I got the next binary that resulted in the a similar error (gh
). Adding all personal tools is one way to get over this problem.LD_LIBRARY_PATH
upon calling other binaries from within the wrapped devenv python.patchelf
/autopatchelf
. In addition,LD_LIBRARY_PATH
should not be used so thatld
won't be forced to use different libraries. See PR https://github.com/cachix/devenv/pull/840 as a poc.To reproduce
Follow the instructions from the readme:
https://gist.github.com/bobvanderlinden/f31299079b67ddcc1b2c6029cf6569f6
Version