search

indygreg / python-build-standalone

Produce redistributable builds of Python
BSD 3-Clause "New" or "Revised" License
1.71k stars 107 forks source link

Restore shared extension module and/or dependency libraries #227

Open indygreg opened 3 months ago

indygreg commented 3 months ago

This project was created in support of PyOxidizer. And a goal of PyOxidizer was to see if it was possible to achieve a single file executable Python application.

In order to achieve that goal, this project built highly statically linked Python distributions.

On Linux and macOS, extension modules are statically linked into libpython. And 3rd party library dependencies of extension modules (OpenSSL, SQLite, etc) are also statically linked into libpython.

This mostly just works. Until it doesn't.

For example, ELF symbols resolution prefers symbols in the global namespace over a local namespace. So if you load a .so providing symbol foo which is also provided by an external libpython.so, the libpython.so symbol is used. This leads to wonkiness like https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html#static-linking-of-libx11-incompatibility-with-pyqt-on-linux.

As the Python distributions in this project have grown a life outside of PyOxidizer, the opinionated choice of statically linking all extension modules and shared libraries is hard to justify. We're foregoing compatibility with aspects of the larger Python ecosystem.

So I'm proposing that we revert to distributing 3rd party libraries and extension modules as shared libraries.

indygreg commented 3 months ago

I looked at this over the weekend. I was soft blocked after running into a bug with patchelf corrupting ELF binaries. I need to read up on how the GNU dynamic loader handles rpath and $ORIGIN before I continue with an alternate solution.