Archspec lacks support for M1 and other apple arm products

[trws]

This came up in a build issue on spack: spack/spack#29735

Archspec identifies an M1, or an a14 or what have you, as a generic aarch64 and tries to supply apple clang with -march=armv8-a, which is both the wrong architecture and an unsupported compiler flag on that compiler. More generally, arm guidance is to never use -march on any compiler, but to use -mcpu instead (completely different handling of these than x86, across all compilers, high-level article here). I'm not sure what the best way is to deal with this is since almost all of the compiler flags listed for arm should probably change, but at a minimum something to handle the apple parts would help.

[giordano]

The host detection on the M1 works for me:

In [2]: archspec.cpu.host()
Out[2]: Microarchitecture('m1', [Microarchitecture('aarch64', [], 'generic', set(), {'gcc': [{'versions': '4.8.0:', 'flags': '-march=armv8-a -mtune=generic'}], 'clang': [{'versions': ':', 'flags': '-march=armv8-a -mtune=generic'}], 'apple-clang': [{'versions': ':', 'flags': '-march=armv8-a -mtune=generic'}], 'arm': [{'versions': ':', 'flags': '-march=armv8-a -mtune=generic'}]}, 0)], 'Apple', set(), {'gcc': [{'versions': '8.0:', 'flags': '-march=armv8.4-a -mtune=generic'}], 'clang': [{'versions': '9.0:', 'flags': '-march=armv8.4-a'}], 'apple-clang': [{'versions': '11.0:', 'flags': '-march=armv8.4-a'}]}, 0)

This is a basic M1, I see the issue you reference mentions M1 Max.

[trws]

The results you're getting are equally incorrect, despite it identifying it as an M1.

[alalazo]

@trws Thanks for the link to the article. The reason why we didn't go with -mcpu in the beginning is that -march can override -mcpu so, if we returned -mcpu, a build-system setting -march would make the flags returned by archspec totally ineffective. On the other hand I was not aware that ARM advice was to use -mcpu, so there's also that.

I wonder what is the best way to proceed here:

1. We can fix the values of -march on GCC and Clang compilers and use -mcpu on Apple Clang if -march is not supported
2. Or, we can use -mcpu everywhere with the caveat that Spack possibly needs to take actions to remove -march flags from compilation on any AArch64 processor

[trws]

I would probably prefer 2 personally, mainly because march just plain doesn't do on arm what it does on x86. That said, if we're going to use option 1, then we should be specifying both -march and -mtune, or the compiler will not necessarily tune for the target otherwise.

Summary as I understand it:

• on x86:
  • march implies both arch and mtune
  • mcpu is just mtune, so it's basically meaningless
• on ARM:
  • march and mtune are orthogonal (because of their heavy use of extensions)
  • mcpu is equivalent to march on x86, implies both arch with all extensions and mtune together

Doing option 2 with the actual target where possible ensures we get all necessary extensions without having to explicitly list them all as well. Why they do this I do not know, but using just -march is almost certainly not what we want from a spack perspective.

[giordano]

As far as I'm concerned for BinaryBuilder.jl, I'd go with option 2. We already don't allow passing -march to our compiler wrappers (and we should probably do the same for -mcpu and -mtune, but those options are less commonly injected by build systems).

However note that -mcpu tuned for a specific CPU may not be available on all versions of a compiler, for this reason in #43 and #44 I left only -mcpu only for the versions of the compilers where the target is actually available. In the other cases, using -march may be a better option, as you can still enable specific features.

[trws]

That's a good point, using -march with the features listed out, preferably paired with -mtune is a whole lot better than nothing for versions where the correct -mcpu value is unavailable.

[wyphan]

To supplement the discussion, here are the slides from John Linford (Arm) that was presented during Ookami User Meeting 2022.

3-SBU Ookami - Linford - Optimize for Arm - Feb22.pdf