Ability to build Python with `--enable-shared`

[toiletsandpaper]

Problem

Rye has a one major disadvantage right now - rye is installing python versions without libpython.so which leads to many problems with using python from other languages.

For example, this https://github.com/astral-sh/rye/issues/920 issue is mostly because of libpython is not available for rye's python.

How it should work

For apt or dnf/yum we have python3-dev and python3-devel to download shared object For pyenv we have https://github.com/pyenv/pyenv/blob/master/plugins/python-build/README.md#building-with---enable-shared

IMO, this feature is very important and is a blocker for most of python developers

[bluss]

Rye's pythons are built with --enable-shared, and installs libpython and the library is there, but I think this is the same issue as #646 - the configuration paths to where those libraries are located are not correct and need to be set after installing the toolchain in its final location.

Trying to find it, we see it is there:

> find .rye/py -iname "*libpython*.so"
.rye/py/cpython@3.8.18/lib/libpython3.so
.rye/py/cpython@3.8.18/lib/libpython3.8.so
.rye/py/cpython@3.11.8/install/lib/libpython3.so
.rye/py/cpython@3.11.8/install/lib/libpython3.11.so
.rye/py/cpython@3.12.1/lib/libpython3.so
.rye/py/cpython@3.12.1/lib/libpython3.12.so
.rye/py/cpython@3.12.2/install/lib/libpython3.so
.rye/py/cpython@3.12.2/install/lib/libpython3.12.so

Unfortunately it doesn't tell the truth about where it is

> python +3.12 -m sysconfig | grep LIBDIR
    LIBDIR = "/install/lib"

This LIBDIR value is what it ships with and needs to be updated.

With the sysconfigpatcher script - which is WIP and only works on linux I think, then it does the right thing

> python +3.11 -m sysconfig | grep LIBDIR
    LIBDIR = "/home/user/.rye/py/cpython@3.11.8/install/lib"
    PLATLIBDIR = "lib"

[dewoller]

Hi @bluss

Rye is great, a breath of fresh air in this messy python library space. Unfortunately, I also need shared library support, and, on mac, the following line comes up empty

find ~/.rye/py -iname "libpython.so*"

There are no libpython* files under the .rye directory, so it appears that the libpython does not get installed on the mac.

Thanks Dennis

[bluss]

I haven't done any further experiments on macos, but the sysconfigpatcher script worked for me on macos, with it I can build something that seems to require the python lib, and libpython3.12.dylib is present - it's not an .so.

[intercepted16]

@bluss Your script doesn't seem to work for me. The environment variables successfully change, but the actual LIBDIR echoed doesn't.

[bluss]

@intercepted16 which exact toolchain have you downloaded? Could you get its name inside ~/.rye/py for me? (And platform + architecture?)

[intercepted16]

@bluss

ll -a ~/.rye/py Permissions Size User Date Modified Name drwxr-xr-x@ - ab 6 Aug 20:42 cpython@3.12.4

I'm on x64 and using Fedora (Linux) KDE 40

[bluss]

@intercepted16 unfortunately it seems to work for me with that version, so more information would be needed. If you can, open an issue to sysconfigpatcher with steps to reproduce?