i686 (cross-compile on amd64): Python3 Build stalls under Chroot

[dslm4515]

Cross-compiling i686 on a AMD64 host.

Under chroot, python3 build configures fine but locks up. A CPU core is stuck at 100% but no activity. Last command executed by the build:

 LD_LIBRARY_PATH=/sources/Python-3.12.2 ./python -E -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform

ctl-c does nothing under chroot. Only way to 'break' the lockup is to terminate the python process from the host:

$ ps -A | grep python
 7804 pts/0    01:49:11 python
$ sudo kill -s 9 7804

When the python process terminates, the chroot session throws an message [but chroot still functional]:

make: *** [Makefile:1144: checksharedmods] Killed

Perhaps, there are tests that need to be skipped?

Per Chimera Linux,

# disable tests that disagree with our build environment or musl
make_check_args = [
    "EXTRATESTOPTS="
    "-i test_chown_* "
    "-i test_getspnam_exception "
    "-i test_find_library_with_* "
    "-i test_localtime_daylight_*_dst_true "
    "-i test__locale "
    "-i test_c_locale_coercion "
    "-i test_cmd_line "
    "-i test_locale "
    "-i test_os "
    "-i test_re "
    "-i test_readline "
    "-i test_threading "
    "-i test_unicodedata "
    "-i test_urllib2net "  # just loops blocked connection failures into success
    "-i test_tools "
]

[dslm4515]

Fixed issue by adding --with-build-python=python3.12 to configure command

[dslm4515]

Build now completes, but now the install locks up. Last command executed:

 LD_LIBRARY_PATH=/sources/Python-3.12.2 \
_PYTHON_PROJECT_BASE=/sources/Python-3.12.2 \
_PYTHON_HOST_PLATFORM= PYTHONPATH=/sources/Python-3.12.2/build/lib.-3.12:./Lib \
_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__linux_i386-linux-musl python3.12 -m ensurepip  \
$ensurepip --root=/BUILD/ ;

[dslm4515]

Perhaps, build python3 without pip [and build it later]? ... yes install now completes

If building later, build python-setuptools before pip

[dslm4515]

Building setuptools to install pip. Install stalls at:

Copying setuptools.egg-info to /BUILD/usr/lib/python3.12/site-packages/setuptools-69.2.0-py3.12.egg-info
running install_script

Looks like the stage 2 (system) python cannot run scripts. Lock up is just like before, because 1 CPU thread locks at 100% use, but nothing is done

[dslm4515]

I did notice that while Python3 was compiled, it was using llvm-as instead of elftoolchain's as.

Perhaps, rebuild python3 to use elftoolchain instead of LLVM's binary tools? i.e. add AR=ar AS=as RANLIB=ranlib STRIP=strip to configure.

I know during building under chroot, stripping with llvm-strip caused stripped binaries to segfault... makes me wonder if all of LLVM base should be rebuilt, and not just clang...

[dslm4515]

No. Python build still stalls even if elftoolchain is used

[dslm4515]

Rebuild python with llvmtools' clang fixes issue!

Old clang config for llvmtools will cause the ssl module to not build. Remove the clang config:

mv /llvmtools/bin/${TUPLE}.cfg /tmp/${TUPLE}.cfg.llvmtools

llvmtools' clang will need a new config so that installed headers are found:

cat > /llvmtools/bin/${TUPLE}.cfg << "EOF"
-I/usr/include
-L/usr/lib
EOF

Now rebuild python:

$ CC=/llvmtools/bin/clang CXX=/llvmtools/bin/clang++ \
./configure --prefix=/usr       \
            --enable-shared     \
            --with-system-expat \
            --with-system-ffi   \
            --with-ensurepip=yes \
            --enable-ipv6 \
            --enable-loadable-sqlite-extensions \
            --with-computed-gotos \
            --enable-ipv6 \
            --with-openssl=/opt/openssl \
            --with-openssl-rpath=/opt/openssl/lib

[dslm4515]

So far all python modules built after python build fine.

Keeping this open to see if there are issues when using BMLFS after this build

[dslm4515]

Created the workaround by using stage 1 clang via commit 8116309b4d4bbd5fbe4a52c7db4d9648d0eae124

[dslm4515]

I built a few python modules with this stage 2 python (build by stage1 clang) with no problems.

Closing.