Incompatible gcc flags on non-AMD64 platforms

[maranov]

Hi, the 2.14 fails to build on non-AMD64 platforms, where some of the (optimization) GCC flags are unavailable. I've managed to build it on ARMv8 by limiting the flags to just AMD64:

 build.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/build.py b/build.py
index 5608858..1512eb4 100644
--- a/build.py
+++ b/build.py
@@ -1,5 +1,6 @@
 import os
 import shutil
+import platform
 import sys
 from distutils.core import Distribution, Extension
 from distutils.command import build_ext
@@ -14,13 +15,16 @@ def build() -> None:
         compile_args += [
             "-march=native",
             "-O3",
-            "-msse",
-            "-msse2",
-            "-mfma",
-            "-mfpmath=sse",
             "-Wno-pointer-sign",
             "-Wno-unused-variable",
         ]
+        if platform.uname()[4] == 'AMD64':
+            compile_args += [
+                "-msse",
+                "-msse2",
+                "-mfma",
+                "-mfpmath=sse",
+                ]

     extensions = [
         Extension(

AFAIK there's no direct equivalent for any of these flags on AArch64 (vectorization is already part of the base instruction set). Running python -m pytest tests passes OK. The make ext/make test instructions from the Readme no longer work due to setup.py being removed.

What do you think? I could open a PR if this looks OK.

Build attempt

``` # pip install . --upgrade Processing /host Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting pyzmq<25.0,>=24.0 Downloading pyzmq-24.0.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (894 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 894.4/894.4 KB 6.0 MB/s eta 0:00:00 Collecting numpy<2.0,>=1.24 Downloading numpy-1.24.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 12.6 MB/s eta 0:00:00 Building wheels for collected packages: pymodes Building wheel for pymodes (pyproject.toml) ... error error: subprocess-exited-with-error × Building wheel for pymodes (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [59 lines of output] Compiling pyModeS/c_common.pyx because it changed. Compiling pyModeS/decoder/flarm/decode.pyx because it changed. [1/2] Cythonizing pyModeS/c_common.pyx [2/2] Cythonizing pyModeS/decoder/flarm/decode.pyx gcc: error: unrecognized command-line option ‘-msse’ gcc: error: unrecognized command-line option ‘-msse2’ gcc: error: unrecognized command-line option ‘-mfma’ gcc: error: unrecognized command-line option ‘-mfpmath=sse’ Traceback (most recent call last): File "/usr/local/lib/python3.8/distutils/unixccompiler.py", line 117, in _compile self.spawn(compiler_so + cc_args + [src, '-o', obj] + File "/usr/local/lib/python3.8/distutils/ccompiler.py", line 910, in spawn spawn(cmd, dry_run=self.dry_run) File "/usr/local/lib/python3.8/distutils/spawn.py", line 36, in spawn _spawn_posix(cmd, search_path, dry_run=dry_run) File "/usr/local/lib/python3.8/distutils/spawn.py", line 157, in _spawn_posix raise DistutilsExecError( distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "build.py", line 77, in build() File "build.py", line 65, in build cmd.run() File "/usr/local/lib/python3.8/distutils/command/build_ext.py", line 340, in run self.build_extensions() File "/usr/local/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions self._build_extensions_serial() File "/usr/local/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial self.build_extension(ext) File "/usr/local/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension objects = self.compiler.compile(sources, File "/usr/local/lib/python3.8/distutils/ccompiler.py", line 574, in compile self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts) File "/usr/local/lib/python3.8/distutils/unixccompiler.py", line 120, in _compile raise CompileError(msg) distutils.errors.CompileError: command 'gcc' failed with exit status 1 Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in main() File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel return _build_backend().build_wheel(wheel_directory, config_settings, File "/tmp/pip-build-env-7aee21lq/overlay/lib/python3.8/site-packages/poetry/core/masonry/api.py", line 56, in build_wheel return WheelBuilder.make_in( File "/tmp/pip-build-env-7aee21lq/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 85, in make_in wb.build(target_dir=directory) File "/tmp/pip-build-env-7aee21lq/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 120, in build self._build(zip_file) File "/tmp/pip-build-env-7aee21lq/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 169, in _build self._run_build_script(self._package.build_script) File "/tmp/pip-build-env-7aee21lq/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 229, in _run_build_script subprocess.check_call([self.executable.as_posix(), build_script]) File "/usr/local/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/usr/local/bin/python', 'build.py']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pymodes Failed to build pymodes ERROR: Could not build wheels for pymodes, which is required to install pyproject.toml-based projects WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available. You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. ```

[xoolive]

Thank you so much! Could you please open a PR and maybe see if you can include your platform in the github set of actions (the tests should be enough)?

[maranov]

Haven't used Actions yet (just GitLab CI/CD). I've experimented a bit with https://github.com/marketplace/actions/run-on-architecture, but I haven't figured out how to combine it with poetry/venv/etc. steps. Maybe I'll experiment a bit more later...

Either way, I've submitted just the build fix. The conditions should work fine based on the tests on my machines:

An old AMD64 Centos 7:

ARMv8 (RPi 4) Ubuntu:

AMD64 Windows 10:

[xoolive]

I think the action you got is a good one, we can include it in the matrix run in .github/workflows/run_tests.yml There should be only one item to add in the whole set of steps, then things should go smoothly.

[maranov]

I've done a bit of experimenting in https://github.com/maranov/pyModeS/tree/feature/aarch64-actions. The install-poetry and other actions don't support different architectures and multiple actions can't be combined (AFAIK). So the solution would IMHO be an additional separate job, where everything runs in an AArch64 container. Without the setup-python/install-poetry I'm limited to the Python version supplied with the distro, though. There's some issue with poetry as well that I haven't figured out yet.

Normally I'd just use the official Python Docker containers, but Actions only support AMD64/x86 and run-on-arch already needs to make its own container to emulate the chosen architecture. So, we'll probably have to accept that non-AMD64 pipeline will be more limited regarding OS/Python version combinations.