cmd/compile: switch to a register-based calling convention for Go functions

aclements commented 4 years ago

I propose that we switch the Go internal ABI (used between Go functions) from stack-based to register-based argument and result passing for Go ~1.16~ 1.17.

I lay out the details of our proposal and the work required in this document.

The ABI specification can be found here.

This was previously proposed in #18597, where @dr2chase did some excellent prototyping work that our new proposal builds on. With multiple ABI support, we’re now in a much better position to execute on this without breaking compatibility with the existing body of Go assembly code. I’ve opened this new issue to focus on discussion of our new proposal.

/cc @dr2chase @danscales @thanm @cherrymui @mknyszek @prattmic @randall77

An incomplete and evolving list of tasks:

[X] Define ABI specification (CL, @aclements)
[X] Add GOEXPERIMENT=regabi and asm macro (CL)
[x] cmd/compile: late call lowering (CLs)
[x] cmd/compile: implement register arguments (@dr2chase)
- [x] spill around morestack call (CL)
- [x] add temporary ABI pragma for testing (CL)
- [x] add abstract argument registers (WIP CL)
- [x] implement ABI rules (CL)
- [x] use abstract argument registers
[x] cmd/compile: implement register returns
- [x] late return lowering (CL, @dr2chase)
- [x] implement ABI rules (CL, @thanm)
[x] cmd/compile: ABI wrappers (CL, old CL)
- [x] cmd/link: mangle wrapper names for ELF (CL)
- [x] cmd/link: mangle wrapper names for PE (CL)
- [x] implement zeroing of X15 in ABI0->ABIInternal wrappers (CL)
[X] cmd/asm: add syntax for runtime packages to reference ABIInternal symbols (CL)
- [x] Mark runtime.{goexit,asyncPreempt}, reflect.{makeFuncStub,methodValueCall} definitions ABIInternal (CL)
- [x] Mark reference to runtime.callbackWrap in sys_windows_amd64.s as ABIInternal (CL)
[x] runtime: fix assembly closure calls (mcall) (@aclements, CL)
[x] reflect: support calling ABIInternal (CL)
[x] reflect: support MakeFunc and method value calls (CL)
[x] go and defer statements
- [x] cmd/compile: desugar go/defer to zero-argument (@thanm, CL)
- [x] cmd/compile: fix defer recover() (@cherrymui, CL)
- [x] cmd/compile: desugar go/defer with results (@cherrymui, CL)
[X] cmd/cgo: decouple cgo callbacks from ABI (CL)
[x] runtime: finalizer support (CL)
[x] runtime: Windows callback support (CL)
[x] cmd/internal/obj: introduce and use RegEntryTmp registers in prologue (@mknyszek, CL)
[x] cmd/compile: generate correct ABI0 argument maps for body-less functions (see WriteFuncMap) (@dr2chase, CL)
[x] cmd/compile: make cgo_unsafe_args generate an ABI0 function (or an ABIInternal-with-no-registers function) (@cherrymui, CL)
[x] runtime: update debug call protocol (CL)
[x] Fix ABI0/ABIInternal confusion in cgo (CL)
[x] Run signature fuzzer and fix bugs (everyone)
[x] Get all.bash to pass on Linux, Windows, and Darwin (everyone)

High-priority non-critical path

[x] Export ABI information across package boundaries to eliminate more wrappers (@aclements, CL)
[x] Make funcPC always return the "native" PC of a function (maybe also introduce ABIOther) (@cherrymui, CL)
[x] Strip unnecessary ABIInternal annotations once funcPC is in
[x] Tracebacks
- [x] Define extended argument metadata format (in progress, @cherrymui)
- [x] cmd/compile: produce extended argument metadata (@cherrymui, CL)
- [x] runtime: consume extended argument metadata (@cherrymui, CL)
- [x] cmd/compile,runtime: traceback metadata for spill slot liveness
[x] DWARF
- [x] cmd/compile: ensure arguments have appropriate DWARF locations
- [x] Ensure GDB and Delve can find register arguments
- ~[ ] cmd/compile: add a DWARF vendor attribute with function argument frame size (context)~
[x] Wrapper mangling for non-critical arches
- [x] cmd/link: mangle wrapper names for Mach-O (CL)
- [x] cmd/link: mangle wrapper names for xcoff
- [x] cmd/link: mangle wrapper names for Plan 9
[x] runtime: simplify go/defer call paths
- [x] assert that go/defer frames are zero sized (CL)
- [x] replace defer reflectcall with a direct closure call (CL)
[x] cmd/internal/obj: reject splittable ABIInternal functions without morestack spill info (e.g., asm functions) because we can't generate a correct morestack path (CL)
[x] cmd/asm: don't reference args_stackmap in ABIInternal functions (because it's for ABI0) (CL)
[x] Port performance-critical assembly to ABIInternal (see CL 296372 for valuable functions) (CL 308931, CL 310184)
[x] math: fix overhead of assembly->Go calls CL 310331
[x] Release notes
- Mention undocumented behavior we've observed applications depending on that may have changed.
- Using reflect.ValueOf(fn).Pointer() to get the PC of an assembly function will now return the PC of the ABI wrapper
- Performance surprise: calling from assembly to Go is slightly more expensive.
- Performance surprise: calling an assembly function via a closure is slightly more expensive.
- Traceback format is much improved

Enabling steps

[x] Enable regabiwrappers by default on amd64 (Linux, Windows, and Darwin)
[x] Enable regabidefer
[x] Enable regabireflect
[x] Enable regabig
[x] Enable regabiargs

Testing

[ ] Function signature fuzzer (ongoing)
- [ ] Static call, closure call, method call, interface call, reflect.{ValueOf(target),MakeFunc(x, target),Method(x)}.{Call,Interface}, called from defer, called from go, called as a finalizer
- [x] {Big,Small} {argument,result} in {memory,registers} and {not addressed,addressed and not leaked to heap,addressed and leaked to heap}
- [x] Use defer in a test function (check arguments, modify results)
- [x] Pass pointers-to-stack and pointers-to-heap
- [x] Cause result to move to heap
- [x] runtime: add MoveStackOnNextCall, assertions for pointer-to-stack/pointer-to-heap, assertions for live/dead (@aclements, CL)
- [x] Integrate runtime testing hook
- [ ] Clean up fuzzer, and get into tools repo.
[x] cmd/compile: revive clobberdead (CL)
[x] cmd/compile: add clobberdeadreg (CL)

Post-MVP

[x] Move g from TLS to register (CL)
- [x] Use g register on non-Linux OSes (see CL)
[x] Reserve and use X15 as zero register (CL)
- ~[ ] Should this be X0 for more compact instruction encoding?~
[ ] Port to other platforms
- [x] amd64
- [x] arm64
- [ ] mips
- [x] ppc
- [ ] s390x
- [x] risc-v
- ~[ ] porting guide~
[x] Eliminate legacy ABI paths for all platforms, even if they're using ABIInternal with 0 registers
[ ] Eliminate spill slots?
[ ] Pass pointer-shaped method receiver in context register (#44827)
[x] runtime: simplify go/defer call paths
- [x] simplify go/defer paths to assume zero args/results
- [x] simplify special _defer allocator and remove special _defer cases from mallocgc (because _defer records will be fixed size).
[ ] cmd/vet: add syntax and checks for register arguments and results in assembly like x+0(FP)
[ ] cmd/vet: check argument size for ABIInternal functions and fix runtime asm declarations

Cleanup (can be done later)

[ ] runtime: port all assembly -> Go calls to ABIInternal?
[x] Eliminate ABIAlias support from cmd/compile, cmd/link, and object format
[x] remove now-redundant fields from ssa.AuxCall
[x] rationalize use of LocalsOffset/FrameOffset (for registers that use a link register, e.g. arm64, powerppc).
[ ] cmd/compile: revisit abiutils.go and clean up the API (also reduce ABIConfig copying)
[ ] cmd/internal/obj: generate stack maps in obj so we can better compact the morestack/body maps and reduce subtlety?
[x] cmd/compile,runtime: re-enable passing arguments to go in runtime (context)
[ ] cmd/compile: always attach ir.Func to function ir.Names (context)
[x] cmd/compile: support ABIInternal for tail calls (context)
[x] cmd/compile: once wrapping of defers is on for all arch/OS combinations, it should be possible to simplify or remove some of the openDefer processing in ssagen (notablly we could get rid of openDeferSave()
~[ ] runtime: think about an ABI-insensitive debugCall~ Delve uses DWARF information
[ ] Fix up named slots & value association
[ ] fix firstpos in ssa.go – sometimes it's late.

gopherbot commented 3 years ago

Change https://golang.org/cl/356519 mentions this issue: cmd/compile/internal: add ABI register information for riscv64

gopherbot commented 3 years ago

Change https://golang.org/cl/357974 mentions this issue: cmd/compile: change context register from X20 to X26

gopherbot commented 3 years ago

Change https://golang.org/cl/357976 mentions this issue: runtime,cmd/compile: change duff{zero,copy} register for regabi

gopherbot commented 2 years ago

Change https://golang.org/cl/359337 mentions this issue: cmd/compile/internal: add ABI register info for riscv64

dr2chase commented 2 years ago

Should I assume riscv register ABI will not make the 1.18 release? That's what it looks like to me, but I want to confirm before bumping this to 1.19.

cherrymui commented 2 years ago

@dr2chase yes.

laboger commented 2 years ago

Currently ABIInternal can only be used with assembler functions in the runtime package. Are there any plans to change that in the future so other assembler functions can take advantage of the register ABI?

cherrymui commented 2 years ago

As the name states, currently ABIInternal is an internal ABI and not a stable one. Therefore it is limited specifically for internal uses. Once we make it (or some revised version of it) a stable ABI, it can be used in assembly code more widely.

(And at that time we'll probably call it ABI1 or something, instead of ABIInternal.)

zhangfannie commented 2 years ago

Currently ABIInternal can only be used with assembler functions in the runtime package. Are there any plans to change that in the future so other assembler functions can take advantage of the register ABI?

I do not think opening ABI1 (when ABIinternal is a stable ABI) to user is a good idea.

The reason is that if ABI1 needs to be improved in the future, in order to consider the compatibility of user assembly code, we may introduce a new ABI, and when it is stable, it will be called ABI2. When we have N ABIs, there are N^2 cases of mutual calls between ABIs, which will be very complicated to maintain in the future.

Also, the purpose of opening register ABI to users is to improve the performance of handwritten assembly. If the overhead of stack-based ABI is high, in this case, you can consider writing a Go program instead of assembly code, and the compiler will generate register-based ABI. If the overhead of handwritten assembly code inself is very high, the overhead of stack-based ABI is negligible.

So I think it is better not to open register ABI to users. Thank you.

cherrymui commented 2 years ago

I think RISCV is done. @mengzhuo could you confirm?

Move the rest to 1.20. (Or perhaps we want to close this and open separate issues for what else we want to do.)

mengzhuo commented 2 years ago

I think RISCV is done. @mengzhuo could you confirm?

Move the rest to 1.20. (Or perhaps we want to close this and open separate issues for what else we want to do.)

Yes, it's done and thank you for your help!

aclements commented 1 year ago

The only tasks remaining here are a handful of clean-ups. We should probably open separate issues to track those and close this, but for the moment I'm just going to move to this 1.21.

mknyszek commented 1 year ago

Since this has been punted a few times, moving this to Backlog. Please comment if you disagree. Thanks!