astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
26.05k stars 762 forks source link

libpython missing #6812

Open CowKeyMan opened 2 months ago

CowKeyMan commented 2 months ago

libpython should be left in the environment. This is useful for us developing c libraries with python bindings. And in fact python expects this as well (look at the not found line below):

(myenv) username@device:/opt/pyenvs$ which python
/opt/pyenvs/myenv/bin/python
(myenv) username@device:/opt/pyenvs$ ldd /opt/pyenvs/myenv/bin/python
        linux-vdso.so.1 (0x00007ffc04b9d000)
        /opt/pyenvs/myenv/bin/../lib/libpython3.11.so.1.0 => not found
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbf79aec000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbf79ae7000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fbf79ae2000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbf799fb000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbf799f4000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbf797cb000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fbf79b0e000)

Right now, it is instead being put in: ${HOME}.local/share/uv/python/cpython-3.11.9-linux-x86_64-gnu/lib, which is great for and speed, but super awful for portability. Maybe there can be a flag for this? I thought the uv venv --relocatable flag would help, but it does not. Maybe a --portable flag would be good for this?

Thanks for the awesome tool otherwise!

frederikschubert commented 2 months ago

We are also encountering the same issue when trying to run pytriton within a uv environment.

smheidrich commented 2 weeks ago

I guess this is only an issue for the python-build-standalone binaries uv uses by default.

If you let uv create the venv with a Python binary from another source, e.g. your system Python installation or one installed via pyenv, it works just fine because they don't have a shared library dependency on libpython*.so:

$ uv venv --python /usr/bin/python3 system-py-venv
...
$ ldd system-py-venv/bin/python
    linux-vdso.so.1 (0x00007ffc870c9000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1233c94000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1233c75000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f1233c4a000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1233a69000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f1233d9f000)

Judging by the sizes of these binaries (python-build-standalone: ~2 MB, pyenv/system: ~30 MB), I guess the latter just link libpython statically.

So maybe uv should somehow find out whether the binary in question has a relative libpython*.so dependency or not and only copy/symlink libpython*.so if so?

But this issue also messes with venvs created with python-build-standalone binaries independently of uv, so here's an issue on their side to maybe do something about it: