search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.4k stars 17.71k forks source link

x/tools/gopls: incorporate pgo into release process #64301

Open findleyr opened 1 year ago

findleyr commented 1 year ago

Reminder issue: we should use pgo with gopls.

We can either calibrate using our benchmarks, or by collecting/merging real editing sessions.

qiulaidongfeng commented 1 year ago

I have looked into optimizing gopls with pgo, but there are few benchmarks in gopls, so it is difficult to measure the performance gains from pgo. I'm curious how to measure the optimization that pgo brings to gopls?

findleyr commented 1 year ago

@qiulaidongfeng there are benchmarks for performance critical operations, across a variety of open-source repositories, defined in the x/tools/gopls/internal/regtest/bench package. These produce the dashboard I linked above, which we monitor to detect performance regressions.

Can you explain why these are inadequate for evaluating pgo? (serious question: perhaps I'm missing something).

qiulaidongfeng commented 1 year ago

The command I use is go test -run=ssss -bench=. -benchtime=500ms ./... The output is this ? golang.org/x/tools/gopls [no test files]

D:\file\gofile\gogit\tools\gopls>go test -run=ssss -bench=. -benchtime=500ms ./... ? golang.org/x/tools/gopls [no test files] ? golang.org/x/tools/gopls/api-diff [no test files] PASS ok golang.org/x/tools/gopls/doc 0.216s ? golang.org/x/tools/gopls/integration/govim [no test files] PASS ok golang.org/x/tools/gopls/internal/astutil 0.402s PASS ok golang.org/x/tools/gopls/internal/bug 0.380s PASS ok golang.org/x/tools/gopls/internal/cmd 0.495s ? golang.org/x/tools/gopls/internal/file [no test files] PASS ok golang.org/x/tools/gopls/internal/goversion 0.190s PASS ok golang.org/x/tools/gopls/internal/hooks 0.242s ? golang.org/x/tools/gopls/internal/immutable [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp 0.222s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/deprecated 0.337s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/embeddirective 0.337s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/fillreturns 0.335s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/fillstruct 0.320s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/infertypeargs 0.332s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/nonewvars 0.337s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/noresultvalues 0.334s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/simplifycompositelit 0.325s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/simplifyrange 0.333s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/simplifyslice 0.331s ? golang.org/x/tools/gopls/internal/lsp/analysis/stubmethods [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/undeclaredname 0.323s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/unusedparams 0.334s ? golang.org/x/tools/gopls/internal/lsp/analysis/unusedparams/cmd [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/unusedvariable 0.332s PASS ok golang.org/x/tools/gopls/internal/lsp/analysis/useany 0.332s ? golang.org/x/tools/gopls/internal/lsp/browser [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/cache 0.236s PASS ok golang.org/x/tools/gopls/internal/lsp/command 0.194s ? golang.org/x/tools/gopls/internal/lsp/command/commandmeta [no test files] ? golang.org/x/tools/gopls/internal/lsp/command/gen [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/debug 0.337s ? golang.org/x/tools/gopls/internal/lsp/debug/log [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/fake 0.303s goos: windows goarch: amd64 pkg: golang.org/x/tools/gopls/internal/lsp/filecache cpu: AMD Ryzen 7 7840HS w/ Radeon 780M Graphics BenchmarkUncontendedGet-16 2324030 253.4 ns/op 32333.17 MB/s BenchmarkUncontendedSet-16 5 117142780 ns/op 69.93 MB/s PASS ok golang.org/x/tools/gopls/internal/lsp/filecache 2.610s PASS ok golang.org/x/tools/gopls/internal/lsp/frob 0.226s PASS ok golang.org/x/tools/gopls/internal/lsp/glob 0.231s PASS ok golang.org/x/tools/gopls/internal/lsp/lru 0.303s PASS ok golang.org/x/tools/gopls/internal/lsp/lsprpc 0.418s ? golang.org/x/tools/gopls/internal/lsp/mod [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/progress 0.105s PASS ok golang.org/x/tools/gopls/internal/lsp/protocol 0.276s PASS ok golang.org/x/tools/gopls/internal/lsp/protocol/generate 0.236s PASS ok golang.org/x/tools/gopls/internal/lsp/regtest 0.338s PASS ok golang.org/x/tools/gopls/internal/lsp/safetoken 0.302s PASS ok golang.org/x/tools/gopls/internal/lsp/snippet 0.198s PASS ok golang.org/x/tools/gopls/internal/lsp/source 0.185s PASS ok golang.org/x/tools/gopls/internal/lsp/source/completion 0.175s ? golang.org/x/tools/gopls/internal/lsp/source/methodsets [no test files] goos: windows goarch: amd64 pkg: golang.org/x/tools/gopls/internal/lsp/source/typerefs cpu: AMD Ryzen 7 7840HS w/ Radeon 780M Graphics BenchmarkBuildPackageGraph-16 4 171149075 ns/op --- BENCH: BenchmarkBuildPackageGraph-16 pkgrefs_test.go:257: loaded 810 packages in 1.240107s pkgrefs_test.go:257: loaded 810 packages in 1.2555198s pkgrefs_test.go:257: loaded 810 packages in 1.2275294s pkgrefs_test.go:257: loaded 810 packages in 1.2527227s PASS ok golang.org/x/tools/gopls/internal/lsp/source/typerefs 6.924s ? golang.org/x/tools/gopls/internal/lsp/source/xrefs [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/template 0.227s ? golang.org/x/tools/gopls/internal/lsp/tests [no test files] PASS ok golang.org/x/tools/gopls/internal/lsp/tests/compare 0.203s ? golang.org/x/tools/gopls/internal/lsp/work [no test files] ? golang.org/x/tools/gopls/internal/pathutil [no test files] PASS ok golang.org/x/tools/gopls/internal/persistent 0.098s 2023/11/21 21:33:39 cloning https://github.com/googleapis/google-cloud-go.git@07da765765218debf83148cc7ed8a36d6e8921d5 into C:\Users\26454\AppData\Local\Temp\gopls-bench2084362444\google-cloud-go 2023/11/21 21:33:39 checking out https://github.com/googleapis/google-cloud-go.git: exec: "/bin/sh": executable file not found in %PATH% exit status 1 FAIL golang.org/x/tools/gopls/internal/regtest/bench 0.491s PASS ok golang.org/x/tools/gopls/internal/regtest/codelens 0.493s PASS ok golang.org/x/tools/gopls/internal/regtest/completion 0.488s PASS ok golang.org/x/tools/gopls/internal/regtest/debug 0.493s PASS ok golang.org/x/tools/gopls/internal/regtest/diagnostics 0.487s PASS ok golang.org/x/tools/gopls/internal/regtest/inlayhints 0.497s PASS ok golang.org/x/tools/gopls/internal/regtest/marker 0.489s PASS ok golang.org/x/tools/gopls/internal/regtest/misc 0.510s PASS ok golang.org/x/tools/gopls/internal/regtest/modfile 0.488s PASS ok golang.org/x/tools/gopls/internal/regtest/template 0.497s PASS ok golang.org/x/tools/gopls/internal/regtest/watch 0.496s PASS ok golang.org/x/tools/gopls/internal/regtest/workspace 0.525s PASS ok golang.org/x/tools/gopls/internal/settings 0.389s PASS ok golang.org/x/tools/gopls/internal/telemetry 0.570s ? golang.org/x/tools/gopls/internal/telemetry/cmd/stacks [no test files] ? golang.org/x/tools/gopls/internal/vulncheck [no test files] ? golang.org/x/tools/gopls/internal/vulncheck/govulncheck [no test files] ? golang.org/x/tools/gopls/internal/vulncheck/osv [no test files] ? golang.org/x/tools/gopls/internal/vulncheck/scan [no test files] PASS ok golang.org/x/tools/gopls/internal/vulncheck/semver 0.230s PASS ok golang.org/x/tools/gopls/internal/vulncheck/vulntest 0.340s ? golang.org/x/tools/gopls/release [no test files] FAIL

qiulaidongfeng commented 1 year ago

@findleyr I didn't see before x/tools/gopls/internal/regtest/bench, because I use of pgo from go1.21 released just a few months. I have been using go test-bench =. Maybe I fell into a mindset, sorry to make you confused.

findleyr commented 1 year ago

Ah, those benchmarks are failing because they use /bin/sh (the perf builders are linux). We can and should fix this.