search

indygreg / python-build-standalone

Produce redistributable builds of Python
BSD 3-Clause "New" or "Revised" License
1.71k stars 107 forks source link

python-config.py has an un-rewritten shebang referencing /install #145

Closed Edward-Knight closed 1 year ago

Edward-Knight commented 1 year ago

Looking at one of the latest Python builds, I found that one of the files contains a shebang that hasn't been re-written.

lib/python3.10/config-3.10-aarch64-linux-gnu/python-config.py has the shebang:

#!/install/bin/python3.10

Where it could be rewritten to:

#!/bin/sh
"exec" "$(dirname $0)/../../../bin/python3.10" "$0" "$@"

This would require modification of the current fix_shebangs.py script - currently it is only run on the bin directory tree and rewrites relative to bin (where the above modification traverses back up the lib directory tree and down into bin again).

https://github.com/indygreg/python-build-standalone/blob/adff2af8605b6d64946d480651d20bc0df3582a1/cpython-unix/build-cpython.sh#L868-L914

I'm not overly familiar with the CPython internals or what lib/python3.10/config-3.10-aarch64-linux-gnu/python-config.py is used for - it currently isn't causing us any issues. Furthermore I did a rough grep to see if the absolute build path was included in any other files, and found several more instances. I imagine many of these are fairly benign, but thought I should bring them to your attention for completeness sake:

Files referencing the directory /install `grep -ir "/install" python` run on `https://github.com/indygreg/python-build-standalone/releases/download/20221002/cpython-3.10.7+20221002-x86_64_v2-unknown-linux-gnu-pgo+lto-full.tar.zst` and then manually filtered false-positives: ``` python/install/bin/python3.10-config:prefix="/install" python/install/bin/python3-config:prefix="/install" python/install/lib/pkgconfig/python3.pc:prefix=/install python/install/lib/pkgconfig/python-3.10.pc:prefix=/install python/install/lib/pkgconfig/python3-embed.pc:prefix=/install python/install/lib/pkgconfig/python-3.10-embed.pc:prefix=/install Binary file python/install/lib/libpython3.10.so matches Binary file python/install/lib/python3.10/test/pstats.pck matches Binary file python/install/lib/python3.10/ensurepip/_bundled/setuptools-63.2.0-py3-none-any.whl matches Binary file python/install/lib/python3.10/ensurepip/_bundled/pip-22.2.2-py3-none-any.whl matches python/install/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile:prefix= /install python/install/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile:CONFIG_ARGS= '--build=x86_64-unknown-linux-gnu' '--host=x86_64-unknown-linux-gnu' '--prefix=/install' '--with-openssl=/tools/deps' '--without-ensurepip' '--with-readline=editline' '--enable-shared' '--enable-optimizations' '--with-lto' 'ac_cv_buggy_getaddrinfo=no' 'ac_cv_file__dev_ptc=no' 'ac_cv_file__dev_ptmx=no' 'build_alias=x86_64-unknown-linux-gnu' 'host_alias=x86_64-unknown-linux-gnu' 'CC=clang' 'CFLAGS=-march=x86-64-v2 -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -I/tools/deps/libedit/include' 'LDFLAGS= -L/tools/deps/lib -Wl,--exclude-libs,ALL -L/tools/deps/libedit/lib' 'CPPFLAGS=-march=x86-64-v2 -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -I/tools/deps/libedit/include' python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'BINDIR': '/install/bin', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'BINLIBDEST': '/install/lib/python3.10', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: "'--host=x86_64-unknown-linux-gnu' '--prefix=/install' " python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'CONFINCLUDEDIR': '/install/include', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'CONFINCLUDEPY': '/install/include/python3.10', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'DESTDIRS': '/install /install/lib /install/lib/python3.10 ' python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: '/install/lib/python3.10/lib-dynload', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'DESTLIB': '/install/lib/python3.10', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'DESTSHARED': '/install/lib/python3.10/lib-dynload', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'INCLDIRSTOMAKE': '/install/include /install/include ' python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: '/install/include/python3.10 /install/include/python3.10', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'INCLUDEDIR': '/install/include', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'INCLUDEPY': '/install/include/python3.10', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'LIBDEST': '/install/lib/python3.10', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'LIBDIR': '/install/lib', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'LIBPC': '/install/lib/pkgconfig', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'LIBPL': '/install/lib/python3.10/config-3.10-x86_64-linux-gnu', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'LOCALMODLIBS': '-L/install/lib -lz -lbz2 -lcrypt -L/tools/deps/lib ' python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'MACHDESTLIB': '/install/lib/python3.10', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'MANDIR': '/install/share/man', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'MODLIBS': '-L/install/lib -lz -lbz2 -lcrypt -L/tools/deps/lib ' python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'SCRIPTDIR': '/install/lib', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'datarootdir': '/install/share', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'exec_prefix': '/install', python/install/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py: 'prefix': '/install', Binary file python/install/lib/libpython3.10.so.1.0 matches python/PYTHON.json: "BINDIR": "/install/bin", python/PYTHON.json: "BINLIBDEST": "/install/lib/python3.10", python/PYTHON.json: "CONFIG_ARGS": "'--build=x86_64-unknown-linux-gnu' '--host=x86_64-unknown-linux-gnu' '--prefix=/install' '--with-openssl=/tools/deps' '--without-ensurepip' '--with-readline=editline' '--enable-shared' '--enable-optimizations' '--with-lto' 'ac_cv_buggy_getaddrinfo=no' 'ac_cv_file__dev_ptc=no' 'ac_cv_file__dev_ptmx=no' 'build_alias=x86_64-unknown-linux-gnu' 'host_alias=x86_64-unknown-linux-gnu' 'CC=clang' 'CFLAGS=-march=x86-64-v2 -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -I/tools/deps/libedit/include' 'LDFLAGS= -L/tools/deps/lib -Wl,--exclude-libs,ALL -L/tools/deps/libedit/lib' 'CPPFLAGS=-march=x86-64-v2 -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -I/tools/deps/libedit/include'", python/PYTHON.json: "CONFINCLUDEDIR": "/install/include", python/PYTHON.json: "CONFINCLUDEPY": "/install/include/python3.10", python/PYTHON.json: "DESTDIRS": "/install /install/lib /install/lib/python3.10 /install/lib/python3.10/lib-dynload", python/PYTHON.json: "DESTLIB": "/install/lib/python3.10", python/PYTHON.json: "DESTSHARED": "/install/lib/python3.10/lib-dynload", python/PYTHON.json: "INCLDIRSTOMAKE": "/install/include /install/include /install/include/python3.10 /install/include/python3.10", python/PYTHON.json: "INCLUDEDIR": "/install/include", python/PYTHON.json: "INCLUDEPY": "/install/include/python3.10", python/PYTHON.json: "LIBDEST": "/install/lib/python3.10", python/PYTHON.json: "LIBDIR": "/install/lib", python/PYTHON.json: "LIBPC": "/install/lib/pkgconfig", python/PYTHON.json: "LIBPL": "/install/lib/python3.10/config-3.10-x86_64-linux-gnu", python/PYTHON.json: "LOCALMODLIBS": "-L/install/lib -lz -lbz2 -lcrypt -L/tools/deps/lib -lffi -ldl -lm -L/tools/deps/lib -lncursesw -L/tools/deps/lib -lpanelw -lncursesw -L/tools/deps/lib -ldb -L/tools/deps/lib -lcrypto -L/tools/deps/lib -llzma -lrt -L/tools/deps/lib -lsqlite3 -lssl -lcrypto -L/tools/deps/lib -ltcl8.6 -ltk8.6 -lX11 -lxcb -lXau -luuid -ledit -lncursesw", python/PYTHON.json: "MACHDESTLIB": "/install/lib/python3.10", python/PYTHON.json: "MANDIR": "/install/share/man", python/PYTHON.json: "MODLIBS": "-L/install/lib -lz -lbz2 -lcrypt -L/tools/deps/lib -lffi -ldl -lm -L/tools/deps/lib -lncursesw -L/tools/deps/lib -lpanelw -lncursesw -L/tools/deps/lib -ldb -L/tools/deps/lib -lcrypto -L/tools/deps/lib -llzma -lrt -L/tools/deps/lib -lsqlite3 -lssl -lcrypto -L/tools/deps/lib -ltcl8.6 -ltk8.6 -lX11 -lxcb -lXau -luuid -ledit -lncursesw", python/PYTHON.json: "SCRIPTDIR": "/install/lib", python/PYTHON.json: "datarootdir": "/install/share", python/PYTHON.json: "exec_prefix": "/install", python/PYTHON.json: "prefix": "/install", ```
indygreg commented 1 year ago

Thanks for the report.

We should definitely fix the bad shebang.

Fixing the other /install references are harder. We have to embed some absolute paths into sysconfig and some other files. Those paths aren't portable across machines let alone outside the container Python was built in! It's an open issue on how to best normalize these values on destination machines, post archive extraction.