tests: test_executable_not_privileged fails on aarch64

[pfilipko1]

Failing tests:

FAILED tests/test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-java-ap-True] - assert 255 == 0
FAILED tests/test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-python-py-spy-True] - assert 255 == 0
FAILED tests/test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-ruby-rbspy-True] - assert 255 == 0
FAILED tests/test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-php-phpspy-True] - assert 255 == 0
FAILED tests/test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-dotnet-dotnet-trace-True] - assert 255 == 0

Example output

gprofiler_exe = PosixPath('/tmp/pytest-of-root/pytest-22/test_executable_not_privileged0/gprofiler'), application_docker_container = <Container: 82a12440dc>, runtime_specific_args = []
assert_collapsed = functools.partial(<function assert_function_in_collapsed at 0xffff9a8215a0>, 'Fibonacci.main'), output_directory = PosixPath('/tmp/pytest-of-root/pytest-22/test_executable_not_privileged0/output')
profiler_flags = ['--no-python', '--no-php', '--no-ruby', '--no-nodejs', '--no-dotnet', '--java-mode=ap'], application_docker_mount = True

    @pytest.mark.parametrize("application_docker_mount", [True])  # adds output_directory mount to the application container
    @pytest.mark.parametrize(
        "runtime,profiler_type",
        [
            ("java", "ap"),
            ("python", "py-spy"),
            ("ruby", "rbspy"),
            ("php", "phpspy"),
            ("dotnet", "dotnet-trace"),
        ],
    )
    @pytest.mark.parametrize(
        "application_docker_capabilities",
        [
            [
                "SYS_PTRACE",                                                                                                                                                                                                                                                                 ]
        ],
    )                                                                                                                                                                                                                                                                                 @pytest.mark.parametrize("in_container", [True])
    def test_executable_not_privileged(
        gprofiler_exe: Path,
        application_docker_container: Container,
        runtime_specific_args: List[str],                                                                                                                                                                                                                                                 assert_collapsed: Callable[[Mapping[str, int]], None],
        output_directory: Path,
        profiler_flags: List[str],
        application_docker_mount: bool,                                                                                                                                                                                                                                               ) -> None:
        """                                                                                                                                                                                                                                                                               Tests gProfiler with lower privileges: runs in a container, as root & with SYS_PTRACE,
        but nothing more.
        """
        os.makedirs(str(output_directory), mode=0o755, exist_ok=True)

        mount_gprofiler_exe = str(output_directory / "gprofiler")
        if not os.path.exists(mount_gprofiler_exe):
            shutil.copy(str(gprofiler_exe), mount_gprofiler_exe)

        command = (
            [
                mount_gprofiler_exe,
                "-v",
                "--output-dir",
                str(output_directory),
                "--disable-pidns-check",  # not running in init PID
                "--no-perf",  # no privileges to run perf
            ]
            + runtime_specific_args
            + profiler_flags
        )
        exit_code, output = application_docker_container.exec_run(cmd=command, privileged=False, user="root:root")

        _no_errors(output.decode())
>       assert exit_code == 0
E       assert 255 == 0
E         +255
E         -0

tests/test_executable.py:132: AssertionError
----------------------------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------------------------
staticx: patchelf returned -6
----------------------------------------------------------------------------------------------------------------------------- Captured stderr setup ------------------------------------------------------------------------------------------------------------------------------
309 INFO: PyInstaller: 4.10
309 INFO: Python: 3.10.6
313 INFO: Platform: Linux-5.19.0-1022-aws-aarch64-with-glibc2.35
315 INFO: UPX is not available.
317 INFO: Extending PYTHONPATH with paths
['/home/ubuntu/gprofiler2', '/app']
759 INFO: checking Analysis
798 INFO: Appending 'datas' from .spec
816 INFO: checking PYZ
840 INFO: checking PKG
844 INFO: Building because toc changed
844 INFO: Building PKG (CArchive) gprofiler.pkg
103049 INFO: Building PKG (CArchive) gprofiler.pkg completed successfully.
103065 INFO: Bootloader /usr/local/lib/python3.10/dist-packages/PyInstaller/bootloader/Linux-64bit-arm/run
103065 INFO: checking EXE
103069 INFO: Rebuilding EXE-00.toc because gprofiler missing
103069 INFO: Building EXE from EXE-00.toc
103070 INFO: Copying bootloader EXE to /tmp/pytest-of-root/pytest-22/test_executable_not_privileged0/gprofiler
103070 INFO: Appending PKG archive to custom ELF section in EXE
104173 INFO: Building EXE from EXE-00.toc completed successfully.
Assertion failed: splitIndex != -1 (patchelf.cc: shiftFile: 504)
================================================================================================================================ warnings summary ================================================================================================================================
test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-java-ap-True]
test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-java-ap-True]
  /usr/local/lib/python3.10/dist-packages/docker/utils/utils.py:52: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    s1 = StrictVersion(v1)

test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-java-ap-True]
test_executable.py::test_executable_not_privileged[True-application_docker_capabilities0-java-ap-True]
  /usr/local/lib/python3.10/dist-packages/docker/utils/utils.py:53: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    s2 = StrictVersion(v2)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

[Jongy]

The failure is in the gprofiler_exe() fixture which builds the exe. This needn't happen in every test, and in any case not related to the test itself.

Can you try running those failing tests with --executable /path/to/gprofiler/exe flag? This way, the fixture is skipped and the test uses the given exe.

Also - you can do it in the same PR - the fixture uses old code to build the exe (pyinstaller + staticx). The pyi.Dockerfile is more complex and runs other logic + checks when invoking pyinstaller and staticx. Let's remove that logic from gprofiler_exe - that is, REQUIRE receiving the precompiled = request.config.getoption("--executable"), instead of checking whether it's None or not, assert it's not None.