cmd/compile: profile-guided optimization

[cherrymui]

We propose adding support for profile-guided optimization (PGO) to the Go gc toolchain. PGO will enable the toolchain to perform application- and workload-specific optimizations based on run-time information. Unlike many compiler optimizations, PGO requires user involvement to collect profiles and feed them back into the build process. Hence, we propose a design that centers user experience and ease of deployment and fits naturally into the broader Go build ecosystem.

Detailed design can be found here.

In summary, we propose

• The go command will search for a profile named default.pgo in the source directory of the main package and, if present, will supply it to the compiler for all packages in the transitive dependencies of the main package. We will also add a -pgo=<path> command line flag to go build that specifies a profile location to use for a PGO build, specifically,
  • -pgo=<path> will select the profile at path,
  • -pgo=auto will select the profile stored in the source directory of the main package (the behavior mentioned above)
  • -pgo=off will turn off PGO entirely
  • For Go 1.20, it will be default to off. In a future release we plan to make it default to auto.
• The content of the profile will be considered an input to the build, and will be incorporated into the build cache key calculation and BuildInfo.
• The compiler will support reading pprof profiles passed in from the go command, and modify its optimization heuristics to use this profile information. Initially we plan to add PGO-based inlining. More optimizations may be added in the future.
• We will initially support pprof CPU profiles, which can be collected through usual means, such as the the runtime/pprof or net/http/pprof packages.

Input welcome. Beyond input on the general approach, we're particularly looking for input on whether PGO should be default enabled in 1.20, and flag and file naming conventions.

If accepted, we plan to implement a preview of PGO in Go 1.20. Raj Barik and Jin Lin plan to contribute their work on the compiler implementation (initial prototype in CL 429863, but not yet following proposed API).

See also previous discussions at #28262. Filing a new issue to make it clearer what we are proposing. Thanks.

cc @aclements @prattmic @rajbarik

[gopherbot]

Change https://go.dev/cl/444557 mentions this issue: sweet: add mode for PGO benchmarking

[gopherbot]

Change https://go.dev/cl/444895 mentions this issue: sweet: rebuild cmd/compile and cmd/link for go-build benchmark

[gopherbot]

Change https://go.dev/cl/446300 mentions this issue: cmd/compile/internal/test: clean up TestPGOIntendedInlining

[gopherbot]

Change https://go.dev/cl/446302 mentions this issue: cmd/compile/internal/pgo: remove ConvertLine2Int

[gopherbot]

Change https://go.dev/cl/446303 mentions this issue: cmd/compile/internal/pgo: remove most global state

[gopherbot]

Change https://go.dev/cl/446755 mentions this issue: cmd/compile/internal/pgo: remove ListOfHotCallSites

[gopherbot]

Change https://go.dev/cl/447016 mentions this issue: cmd/compile: use CDF to determine PGO inline threshold

[gopherbot]

Change https://go.dev/cl/447015 mentions this issue: cmd/compile: use edge weights to decide inlineability in PGO

[gopherbot]

Change https://go.dev/cl/446977 mentions this issue: cmd/compile/internal/pgo: unexport local types and fields

[gopherbot]

Change https://go.dev/cl/446976 mentions this issue: cmd/compile/internal/inline,cmd/compile/internal/pgo: remove candHotNodeMap and candHotEdgeMap

[gopherbot]

Change https://go.dev/cl/447135 mentions this issue: runtime: yield in goschedIfBusy if gp.preempt

[gopherbot]

Change https://go.dev/cl/447315 mentions this issue: cmd/compile/internal/pgo: match on call line offsets

[gopherbot]

Change https://go.dev/cl/447997 mentions this issue: DO NOT SUBMIT: tool for parallel PGO testing on gomotes

[gopherbot]

Change https://go.dev/cl/449500 mentions this issue: DO NOT SUBMIT: experiemntal pprof trimming tool

[lkarlslund]

Excellent discussion here, and this initiative is really great overall. Reading the above, I think @klauspost has some very valid concerns, and I know how much effort he puts into making things perform really well.

I think that some pragmas (forced inlining hints) would be a huge help for package maintainers, and looking at older discussions it was denied with the argument that you could just "copy and paste code" to do the inlining yourself, which to me sounds absurd. Defining functions and methods is about clarity and human understanding, a layer that is removed at compile time, to make efficient execution the primary concern. There is a clash here, which could be improved upon.

My use case is primarily as app developer, but I have one or two modules that others use. I often do profiling of my primary application, which does some heavy reading, decompression, interning and correlations ... but that's just one profile. Another run might be to create data, not to read it. So are multiple profiles supported, and how are they prioritized?

For module developers, there is no reason not to think that they are actually the specialists on what performs or not, and not allowing them to "chip in" with knowledge (either by manual inline pragma hints or by supplying profiling data to be merged at compile time) would be a shame.

[cherrymui]

So are multiple profiles supported, and how are they prioritized?

Sorry, I'm not sure I really understand your question. I'll try to answer as I understand it. For multiple profiles for the same program, user can specify which profile to use in the command line (go build -pgo=<profile>). If a program has different workloads, only the user would know what workloads to optimize for in a particular build. Or, if one wants to build a single binary that works reasonably well for different workloads, they can merge the profiles before the build.

For module developers, there is no reason not to think that they are actually the specialists on what performs or not, and not allowing them to "chip in" with knowledge (either by manual inline pragma hints or by supplying profiling data to be merged at compile time) would be a shame.

Module/library profiles are on our road map, and we definitely want to support it in the future. It is just that it still requires more thinking and designing, so not done in Go 1.20. Thanks for understanding.

[gopherbot]

Change https://go.dev/cl/453636 mentions this issue: doc/go1.20: add release notes for PGO

[gopherbot]

Change https://go.dev/cl/463684 mentions this issue: _content/doc: add PGO usage guide

[aclements]

A preview of PGO support will be available in Go 1.20 (to be released very soon). :tada: The user guide is available here.

We have a few more problems to tackle before we're ready to call PGO generally available, which we're planning to work on for 1.21:

• [x] https://github.com/golang/go/issues/58099
• [x] Make -pgo=auto the default
• [ ] https://github.com/golang/go/issues/58102
• [ ] https://github.com/golang/go/issues/58153

Then the next priority is tackling the long list of further PGO-based optimizations.

[gopherbot]

Change https://go.dev/cl/464477 mentions this issue: _content/doc/go1.20: add link to PGO user guide

[gopherbot]

Change https://go.dev/cl/474236 mentions this issue: cmd/go: enable -pgo=auto by default

[aclements]

I've filed a new umbrella issue to track PGO optimization opportunities: https://github.com/golang/go/issues/62463

[alexanius]

I read the discussion of some optimizations that need the basic block counters (unrolling, pgo register allocation, basic block ordering), but currently I do not see any issues or PRs or discussion of this feature. Is there any proposal or plans for implementing it?

[aclements]

It's in the list of PGO optimization opportunities we'd like to pursue, but we don't yet have specific plans. We likely need discriminators in profiles (#59612) before we can do basic block-level PGO optimizations.