Add support to release Linux aarch64 wheels

[odidev]

Problem

On aarch64, ‘pip install numcodecs’ builds the wheels from source code and then installs it. It requires the user to have a development environment installed on his system. Also, it takes some time to build the wheels than downloading and extracting the wheels from pypi.

Resolution

On aarch64, ‘pip install numcodecs’ should download the wheels from pypi

@joshmoore and Team Please let me know your interest in releasing aarch64 wheels. I can help in this.

[odidev]

I have done some modifications to build the wheel for arm64 but were getting below error -

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/zlib-1.2.8 -Ic-blosc/internal-complibs/zstd-1.4.8 -Ic-blosc/internal-complibs/lz4-1.9.3 -Ic-blosc/internal-complibs/zstd-1.4.8/dictBuilder -Ic-blosc/internal-complibs/zstd-1.4.8/common -Ic-blosc/internal-complibs/zstd-1.4.8/decompress -Ic-blosc/internal-complibs/zstd-1.4.8/legacy -Ic-blosc/internal-complibs/zstd-1.4.8/dll -Ic-blosc/internal-complibs/zstd-1.4.8/deprecated -Ic-blosc/internal-complibs/zstd-1.4.8/compress -Ic-blosc/internal-complibs/zstd-1.4.8/dll/example -I/opt/_internal/cpython-3.6.12/include/python3.6m -c numcodecs/blosc.c -o build/temp.linux-aarch64-3.6/numcodecs/blosc.o -mno-sse2 -mno-avx2
    gcc: error: unrecognized command line option ‘-mno-sse2’
    gcc: error: unrecognized command line option ‘-mno-avx2’
    [numcodecs] command 'gcc' failed with exit status 1
    Traceback (most recent call last):
      File "/opt/_internal/cpython-3.6.12/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
        extra_postargs)
      File "/opt/_internal/cpython-3.6.12/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/opt/_internal/cpython-3.6.12/lib/python3.6/distutils/spawn.py", line 36, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/opt/_internal/cpython-3.6.12/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
        % (cmd, exit_status))
    distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

I have bypassed the flag by modifying the code and now the wheel is building successfully for arm64.

Commit Link - https://github.com/odidev/numcodecs/commit/5a7e2b513e61e77f596f404263aef4644cd9fdf8

Github actions Link - https://github.com/odidev/numcodecs/runs/3789990435?check_suite_focus=true

If the changes looks good. Shall I procced by raising the PR?

[joshmoore]

Hi @odidev. Wow. Thanks for looking into this. (also TIL about [auto]. Nice!) From my side, I'd love to see a PR.

[dstansby]

There's a PR for this here: https://github.com/zarr-developers/numcodecs/pull/315

[dstansby]

building the wheels using emulation is taking a lot of time (hours). Before we go any further or merge support for aarch64 wheels, it would be good to understand:

1. What systems/machines out in the wild use this architechture? Is it common?
2. How long does it take to build numcodecs from source on this system.

If it is fairly quick (~minutes) and these systems are rare, then I'm not sure whether this is worth pursuing. But if they are not rare, and/or builds still take forwever, we can think more about this.