cmd/compile: design and implementation of Profile-Guided Optimization (PGO)

[jinlin-bayarea]

This proposal provides the detailed design and implementation of Profile-Guided Optimization (PGO) in the Go compiler. It augments the design and implementation aspects of PGO with the high-level issue related to PGO.

Background: Inefficiencies in Go programs can be isolated via profiling tools such as pprof and linux profiler perf. Such tools can pinpoint source code regions where most of the execution time is spent. Unlike other optimizing compilers such as LLVM, the Go compiler does not yet perform Profile-Guided Optimization(PGO). PGO uses information about the code’s runtime behavior to guide compiler optimizations such as inlining, code layout etc. PGO can improve application performance in the range 15-30% [LLVM, AutoFDO]. In this proposal, we extend the Go compiler with PGO.

In this proposal, we incorporate the profiles into the frontend of the compiler to build a call graph with node & edge weights (called WeightedCallGraph). The Inliner subsequently uses the WeightedCallGraph to perform profile-guided inlining which aggressively inlines hot functions. We introduce a profile-guided code specialization pass that is tightly integrated with the Inliner and eliminates indirect method call overheads in hot code paths. Furthermore, we annotate IR instructions with their associated profile weights and propagate these to the SSA-level in order to facilitate profile-guided basic-block layout optimization to benefit from better instruction-cache and TLB performance. Finally, we extend Go's linker to also consume the profiles directly and perform function reordering optimization across package boundaries -- which also helps instruction-cache and TLB performance.

The format of the profile file consumed by our PGO is identical to the protobuf format produced by the pprof tool. This format is rich enough to carry additional hardware performance counter information such as cache misses, LBR, etc. Existing perf_data_converter tool from Google can convert a perf.data file produced by the Linux perf into a profile.proto file in protobuf format.

The first version of the code that performs profile-guided inlining is available here. In summary, we introduce the following flags to the go compiler in our first released version:

**-profileuse <filename>**: filename corresponds to protobuf CPU profile. This flag will build the WeightedCallGraph and use it to perform profile-guided inlining.
**-inlinehotthreshold <string_float>** and **-inlinehotbudget <int>**: These two flags are optional as they have been provided with default values. In advanced settings, these flags can be tuned for controlling code size and performance.

Other PGO optimizations such as code specialization, basic block reordering, and function reordering across packages will be open-sourced in subsequent Go compiler releases.

Detailed design document cam be found here (https://go-review.googlesource.com/c/proposal/+/430398/1/design/55025-pgo-design.md)

[gopherbot]

Change https://go.dev/cl/430398 mentions this issue: design/55025-pgo-design.md: add pgo design doc

[erifan]

Is this a duplicate of https://github.com/golang/go/issues/55022 ?

[mvdan]

@erifan not as written; see the first paragraph:

It augments the design and implementation aspects of PGO with the high-level issue related to PGO.

Though I agree that it is unconventional to have two proposals. Usually we would have one proposal issue and one detailed proposal document, instead of two issues and one document.

[ianlancetaylor]

It's fine to have an issue to discuss the design and implementation, but I'm not sure why it has to be a proposal. @jinlin-bayarea is there an API change here that is not covered by #55022? Thanks.

[jinlin-bayarea]

We are still learning the process of upstreaming our design and code changes to Go. Apologies. There is no API change. We have asked our Google collaborators Austin Clement, Cherry Mui, and Michael Pratt to guide us in the right direction. Thank you.

On Tue, Sep 13, 2022 at 7:28 PM Ian Lance Taylor @.***> wrote:

It's fine to have an issue to discuss the design and implementation, but I'm not sure why it has to be a proposal. @jinlin-bayarea https://github.com/jinlin-bayarea is there an API change here that is not covered by #55022 https://github.com/golang/go/issues/55022? Thanks.

— Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/55025#issuecomment-1246153520, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALGEZHNN2VK5QQD2G363ZRDV6EZ6XANCNFSM6AAAAAAQKZHQCM . You are receiving this because you were mentioned.Message ID: @.***>

[rajbarik]

@ianlancetaylor @aclements any recommendation on where to upload our design and implementation proposal? It is currently under review at https://go-review.googlesource.com/c/proposal/+/430398/1/design/55025-pgo-design.md. Thanks.

[rsc]

We asked Uber to send a design for their PGO to the proposal repo for reference in the Go pgo implementation (#55022). This issue is not necessary to submit that CL, so removing from proposal process.

[aclements]

@rajbarik , that's the right place for your design doc. Once you've incorporated the proofreading comments and it's +2'd, we can submit it to the design repo.

[rsc]

Removed from the proposal process. This was determined not to be a “significant change to the language, libraries, or tools” or otherwise of significant importance or interest to the broader Go community. — rsc for the proposal review group

[rajbarik]

Thanks. It is up in gerrit.

[prattmic]

Closing in favor of #55022, which https://go.dev/cl/430398 now references.