patch not working with devbox venv

[shot-codes]

Hey there, I have a python venv created with devbox (don't think this should make much of a difference as it is a regular venv, devbox just creates it.) I launch a shell with the fix-python tool and attempt to patch the venv with fix-python --venv .devbox/virtenv/python/.venv. In response I get the following before successful exit:

Automatically adding "file" to PATH.
Automatically adding "patchelf" to PATH.
Found 1 binary files
Patching file: 
patchelf: getting info about '': No such file or directory
patchelf: getting info about '': No such file or directory
patchelf: getting info about '': No such file or directory

When I try to run my python code with my venv activated I get the following: ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

[shot-codes]

See the same behaviour when just creating a virtualenv without devbox. I'm on NixOS 24.11.20240831.12228ff (Vicuna) x86_64

[GuillaumeDesforges]

Hi! I don't know about devbox, so the help I can provide here is limited unfortunately.

That being said, it looks like it just can't find binary files in the path of the venv.

You may want to check that all binaries are actually located in the .devbox/virtenv/python/.venv, and that there is no symbolic link in there (and there is no symbolic link in the path as well).

You can play with the bash expression to try to understand what's going on:

find "$(realpath ".devbox/virtenv/python/.venv")" -type f -executable -exec sh -c "file -i '{}' | grep -qE 'x-(.*); charset=binary'" \; -print

[shot-codes]

Thanks for the response. Looks like you are correct, the binaries are sym links:

./.venv on  os-agnostic-bash-scripts [?] 
❯ l bin 
total 48K
drwxr-xr-x 1 shot users  164 Sep  3 13:42 .
drwxr-xr-x 1 shot users   56 Sep  3 13:42 ..
-rw-r--r-- 1 shot users 2.1K Sep  3 13:42 activate
-rw-r--r-- 1 shot users  946 Sep  3 13:42 activate.csh
-rw-r--r-- 1 shot users 2.2K Sep  3 13:42 activate.fish
-rw-r--r-- 1 shot users 8.9K Sep  3 13:42 Activate.ps1
-rwxr-xr-x 1 shot users  270 Sep  3 13:42 pip
-rwxr-xr-x 1 shot users  270 Sep  3 13:42 pip3
-rwxr-xr-x 1 shot users  270 Sep  3 13:42 pip3.12
lrwxrwxrwx 1 shot users   10 Sep  3 13:42 python -> python3.12
lrwxrwxrwx 1 shot users   10 Sep  3 13:42 python3 -> python3.12
lrwxrwxrwx 1 shot users   73 Sep  3 13:42 python3.12 -> /nix/store/04gg5w1s662l329a8kh9xcwyp0k64v5a-python3-3.12.4/bin/python3.12

I will try to figure out how I went about making the virtual environments previously and do that, if you have other suggestions they are much appreciated :)

[shot-codes]

Even when I create a virtualenv with the nixpkg python312 the binaries are symlinks. How can I create a venv that actually contains the binaries? Or is there a way to patch with the symlinks

[GuillaumeDesforges]

Ah right, I always pass --copies

  --copies              Try to use copies rather than symlinks, even when
                        symlinks are the default for the platform.

https://docs.python.org/3/library/venv.html

This ensure the binary is an actual file in the venv.

Can you please try passing --copies to venv when creating the venv?

[shot-codes]

Thank you, just came across this myself. The patching seems to work now but I still run into import errors with libstdc++.so:

❯ fix-python --venv .venv                       
Automatically adding "file" to PATH.
Automatically adding "patchelf" to PATH.
Found 3 binary files
Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python

Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python3

Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python3.12

    from . import _zmq
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

I source the venv before and am relatively certain it is not using some othre python version as I don't have python installed globally and if I run the program before sourcing is reports that python is not found.

[GuillaumeDesforges]

Which command do you run that leads to this error? Do you install Python packages as well, or are you using Python without additional packages?

Can we make sure you use the right executable?

which python

What is the output of this?

python --version

What is the output of ldd?

ldd /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python

[shot-codes]

Python is being used in a bash script and it is installing packages. I will look closer at the script now. It has worked in the past with fix-python. In the meantime the outputs:

(.venv)
[nix-shell:~/Develop/skypuzzler/puzzler-system]$ which python
/home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python

(.venv) 
[nix-shell:~/Develop/skypuzzler/puzzler-system]$ python --version
Python 3.12.5

(.venv) 
[nix-shell:~/Develop/skypuzzler/puzzler-system]$ ldd /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python
    linux-vdso.so.1 (0x00007ffcf6588000)
    libpython3.12.so.1.0 => /nix/store/pgb120fb7srbh418v4i2a70aq1w9dawd-python3-3.12.5/lib/libpython3.12.so.1.0 (0x00007f93c9600000)
    libdl.so.2 => /nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libdl.so.2 (0x00007f93c9ecc000)
    libm.so.6 => /nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libm.so.6 (0x00007f93c9de6000)
    libgcc_s.so.1 => /nix/store/zx9zgvy2nsxcmpn0fzx277m5lk352cdf-gcc-13.3.0-lib/lib/libgcc_s.so.1 (0x00007f93c9dc1000)
    libc.so.6 => /nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6 (0x00007f93c9409000)
    /nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/ld-linux-x86-64.so.2 => /nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib64/ld-linux-x86-64.so.2 (0x00007f93c9ed3000)

[shot-codes]

Oh I see what might be happening, the scripts is using python to create a virtualenv which it then uses. So the new venv may not be patched? That being said, as a sanity check I jused the python virtualenv that I did patch (the one with the outputs above) and use pip to install zmq. I then launch the python REPL and attempt to import it but am still met with ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory.

I notice the output of ldd doesn't containe libstdc++, not sure if thats relevant though

[shot-codes]

Tried adding a libs.nix file with the following:

let pkgs = import (builtins.getFlake "nixpkgs") { };
in [
pkgs.stdenv.cc.cc.lib
]

Then patch again and get

Found 6 binary files
Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/lib/python3.12/site-packages/pyzmq.libs/libsodium-1b1f72d5.so.26.1.0
patchelf: cannot find section '.interp'. The input file is most likely statically linked

Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/lib/python3.12/site-packages/pyzmq.libs/libzmq-a430b4ce.so.5.2.5
patchelf: cannot find section '.interp'. The input file is most likely statically linked

Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/lib/python3.12/site-packages/zmq/backend/cython/_zmq.cpython-312-x86_64-linux-gnu.so
patchelf: cannot find section '.interp'. The input file is most likely statically linked

Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python

Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python3

Patching file: /home/shot/Develop/skypuzzler/puzzler-system/.venv/bin/python3.12

*My sanity check works now though, and I can import zmq. Script still doesn't work but I assume thats because the aforementioned creation of another venv. I will try to workaround that.

[GuillaumeDesforges]

Interesting. Does it work only with pkgs.stdenv.cc.cc.lib?

Default is using:

https://github.com/GuillaumeDesforges/fix-python/blob/0e46f1424e37b2ee340fa1163dc9018cb8d88330/fix-python#L10-L15

The .lib part should not be necessary because of makeLibraryPath

https://github.com/GuillaumeDesforges/fix-python/blob/0e46f1424e37b2ee340fa1163dc9018cb8d88330/fix-python#L118