runtime: binaries should fail at startup when built with a GOARM64 version not supported on the runtime hardware

cespare commented 2 months ago

Go version

go version go1.23.0 linux/arm64

Output of `go env` in your module/workspace:

Not relevant

What did you do?

On an arm64 host, I ran

GOARM64=v9.5 go test

on a simple hello world test.

What did you see happen?

The test ran and passed.

What did you expect to see?

This is on an AWS c7g instance. This is a graviton3 chip. I'm not 100% sure what the exact arm64 architecture version of that chip is (anyone know a good way to tell?), but I'm pretty sure it's v8.x. Someone on Hacker News claims that graviton3 is v8.4. In /proc/cpuinfo I see

processor       : 0
BogoMIPS        : 2100.00
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0xd40
CPU revision    : 1

Anyway, I'm pretty sure that it does not support arm64 v9.5. Yet, the binary I compiled with GOARM64=v9.5 does not check for CPU capabilities and exit on startup.

By contrast, on my local machine (AMD Ryzen 9 3900X) if I run

$ GOAMD64=v4 go test
This program can only be run on AMD64 processors with v4 microarchitecture support.
exit status 1

because my chip only supports GOAMD64=v3.

From the CL 559555 it sounds like GOARM64 support isn't being used in the compiler to do anything yet. However, it's still important that we fix this, because the arm64.vX.Y build tags are available to user code. I am able to write code today using (for example) the arm64.v9.3 build tag to guard some v9.3-specific feature; when I run this binary on a v8.0 CPU it won't crash on startup but will hit some invalid instruction later.

P.S. It would be good if someone would update some of the wiki pages for GOARM64; in particular, MinimumRequirements and GoArm. I found it a little hard to get info about GOARM64; as far as I can tell, besides the Go 1.23 release notes, the only places it's mentioned are go help environment and go help buildconstraint.

/cc @andreybokhanko @cherrymui

gabyhelp commented 2 months ago

Related Issues and Documentation

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

andreybokhanko commented 2 months ago

@cespare , thanks for reporting this!

I'm the original author of GOARM64 feature, so it's appropriate for me to fix this issue. In general, I fully agree with @cespare proposal -- we should do the same on ARM64 what we do on AMD64, if only for the sake of consistency.

@cherrymui , could you, please, assign this on me?

gopherbot commented 2 months ago

Change https://go.dev/cl/609855 mentions this issue: GoArm, MinimumRequirements: add documentation on GOARM64

gopherbot commented 2 months ago

Change https://go.dev/cl/610195 mentions this issue: runtime: Check ARM64 architecture at runtime

andreybokhanko commented 2 months ago

CL with a fix submitted for review: https://go-review.googlesource.com/c/go/+/610195

Note that it only checks presence of LSE extension if compilation targeted it (which is true for all ARM64>=v8.1). Why not all other extensions? Two reasons:

At the moment, we only use (and pass as a macro to assembler) LSE. Adding a check for other ARM64 extensions (that we don't actually use at the moment) would require adding / passing a slew of additional macros.
The motivation for AMD64 check is program crashing ungracefully when executed on an older CPU (https://github.com/golang/go/issues/49586). Thus, AMD64 runtime checks presence of instruction that can actually be generated and used for newer CPUs targeted by GOAMD64, not all possible AMD64 extensions.

cespare commented 2 months ago

@andreybokhanko Let me highlight something I mentioned in the original report:

Compiling with a particular GOARM64 value does not only have an effect on the instructions generated by the Go compiler. It also is made available to user code via the arm64.vX.Y build tags. That means that users can write assembly code which targets particular arm64 features and guard it with build tags. Even if the compiler never generates some particular v9.5 instruction, I can write some code in a //go:build arm64.v9.5 file which uses that instruction. The resulting binary should check for v9.5 support when it starts up so that that instruction is known to be valid for the chip it's running on.

cherrymui commented 2 months ago

I also think that checking the required version and features would be better.

What is the best way to check the CPU version and features? From https://go-review.googlesource.com/c/go/+/610195/comment/219aedb1_f5d5dd4e/ it sounds like we can only check individual features, and it would be pretty complex to check the version?

andreybokhanko commented 2 months ago

@andreybokhanko Let me highlight something I mentioned in the original report:

Compiling with a particular GOARM64 value does not only have an effect on the instructions generated by the Go compiler. It also is made available to user code via the arm64.vX.Y build tags. That means that users can write assembly code which targets particular arm64 features and guard it with build tags. Even if the compiler never generates some particular v9.5 instruction, I can write some code in a //go:build arm64.v9.5 file which uses that instruction. The resulting binary should check for v9.5 support when it starts up so that that instruction is known to be valid for the chip it's running on.

OK, got it.

Note, though, that while build tags with an arch version number can be used in theory, in practice this happens extremely rare -- for example, there is just one case in the entire Go compiler + runtime source code base that checks for a specific version of amd64 build tag. Check for amd64 tag, without a specific version, is widespread.

I'm not saying that checking for all possible ARM64 versions + extensions would be a bad thing -- just that in case of ARM64 it's quite cumbersome and error-prone see my comment for details) while relatively low in importance.

cespare commented 2 months ago

Note, though, that while build tags with an arch version number can be used in theory, in practice this happens extremely rare -- for example, there is just one case in the entire Go compiler + runtime source code base that checks for a specific version of amd64 build tag. Check for amd64 tag, without a specific version, is widespread.

Sure, it's not a super common need. But I don't think it will be that uncommon -- if you are going to the trouble of writing assembly, it is often because you want to squeeze as much performance as possible out of the hardware for a specialized tasks, and using newer instructions on newer hardware is one aspect of that.

The amd64 build tag has been around forever, but the amd64.vX build tags were only added two years ago in Go 1.20 (see #45454). People haven't had that much time to start using them yet. At my work, we only started using it in the last year.

golang / go