/usr/src/mariner/BUILD/module_info.ld missing

[christian-vorhemus]

I'm using Azure Percept November Service Release (2111) which is using the following Mariner version in their latest update:

root [ /home/admin ]# cat /etc/*-release
DISTRIB_ID="Mariner"
DISTRIB_RELEASE="1.0.20211106"
DISTRIB_CODENAME=Mariner
DISTRIB_DESCRIPTION="CBL-Mariner 1.0.20211106"
CBL-Mariner 1.0.20211106
MARINER_BUILD_NUMBER=3060a05
NAME="Common Base Linux Mariner"
VERSION="1.0.20211106"
ID=mariner
VERSION_ID=1.0
PRETTY_NAME="CBL-Mariner/Linux"
ANSI_COLOR="1;34"
HOME_URL="https://aka.ms/cbl-mariner"
BUG_REPORT_URL="https://aka.ms/cbl-mariner"
SUPPORT_URL="https://aka.ms/cbl-mariner"

When I try to install the this Python module running sudo pip3 install ./azure-percept-py/ I get

    aarch64-unknown-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/mariner/default-hardened-cc1 -fasynchronous-unwind-tables -fstack-clash-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/mariner/default-hardened-cc1 -fasynchronous-unwind-tables -fstack-clash-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/mariner/default-hardened-cc1 -fasynchronous-unwind-tables -fstack-clash-protection -fPIC -I/usr/include/python3.7m -c iot/percept/ext/perceptmodule.c -o build/temp.linux-aarch64-3.7/iot/percept/ext/perceptmodule.o
    aarch64-unknown-linux-gnu-gcc -pthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/mariner/default-hardened-ld -Wl,-dT,/usr/src/mariner/BUILD/module_info.ld -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/mariner/default-hardened-ld -Wl,-dT,/usr/src/mariner/BUILD/module_info.ld build/temp.linux-aarch64-3.7/iot/percept/ext/perceptmodule.o -L/usr/lib -lpython3.7m -o build/lib.linux-aarch64-3.7/_hardware.cpython-37m-aarch64-linux-gnu.so -lasound -lpthread
    /bin/ld: cannot open linker script file /usr/src/mariner/BUILD/module_info.ld: No such file or directory
    collect2: error: ld returned 1 exit status
    error: command 'aarch64-unknown-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-93s_h22g/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-93s_h22g/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-0q394ro1/install-record.txt --single-version-externally-managed --compile --install-headers /usr/include/python3.7m/azure-percept Check the logs for full command output.

The same seems to be true for other Python modules that require compilation + linking like sudo pip3 install psutil:

    aarch64-unknown-linux-gnu-gcc -pthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/mariner/default-hardened-ld -Wl,-dT,/usr/src/mariner/BUILD/module_info.ld -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/mariner/default-hardened-ld -Wl,-dT,/usr/src/mariner/BUILD/module_info.ld build/temp.linux-aarch64-3.7/psutil/_psutil_common.o build/temp.linux-aarch64-3.7/psutil/_psutil_posix.o build/temp.linux-aarch64-3.7/psutil/_psutil_linux.o -L/usr/lib -lpython3.7m -o build/lib.linux-aarch64-3.7/psutil/_psutil_linux.cpython-37m-aarch64-linux-gnu.so
    /bin/ld: cannot open linker script file /usr/src/mariner/BUILD/module_info.ld: No such file or directory
    collect2: error: ld returned 1 exit status
    error: command 'aarch64-unknown-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_eyvt981/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_eyvt981/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-49lb9bla/install-record.txt --single-version-externally-managed --compile --install-headers /usr/include/python3.7m/psutil Check the logs for full command output.

I'm not sure where module_info.ld is coming from and why Python requires this linker script but it worked in previous versions of Mariner (unfortunately I don't have the number of the working version).

[elsaco]

@christian-vorhemus you might have to first run /usr/lib/rpm/mariner/gen-ld-script.sh to generate /usr/src/mariner/BUILD/module_info.ld. This should get you past the no such file or directory error.

[christian-vorhemus]

Indeed, that fixes the issue! Thanks @elsaco! Just out of curiosity, do you know why it is designed that way? Why is there a script to generate a linker script?

[ghost]

Thanks for bringing up the issue! Upon further investigation, it looks like build flags that only make sense in our build environment are leaking out via Python's sysconfig module.

I've opened up #1824 as a fix for our 2.0 branch. Once merged, I will backport this to our 1.0 branch when time permits.

And to answer your other question- we generate a linker script at build-time to insert distro/package information into ELF objects. This ensures the provenance of a package is included in a core dump. This is part of a cross-distro effort, and you can find out more about it here.

[ddstreetmicrosoft]

this appears fixed per the last comment, closing