search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.55k stars 17.61k forks source link

x/tools/cmd/guru: implements can take extreme amount of resources (e.g. when run from vscode) depending on working directory #41243

Closed Dieterbe closed 1 month ago

Dieterbe commented 4 years ago

Hello, first of all, thanks for the work that's been put into guru. It's certainly been quite helpful to me over the years. I know guru is no longer actively maintained (and I don't expect a resolution), but considering gopls is still in alpha, i thought it might be helpful to others (or to my future self) to document this issue.

What version of Go are you using (go version)?

go version go1.15.1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dieter/.cache/go-build"
GOENV="/home/dieter/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/dieter/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/dieter/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/dieter/go-build095063144=/tmp/go-build -gno-record-gcc-switches"

I found out that based on the working directory when guru implements is run, it can take an extreme amount of cpu and take a very long time.

In particular, consider these 3 types of runs:

1) cd /home/dieter && \
    guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
2) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
3) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'

only 1) returns in a reasonable amount of time. 2) is how visual studio code invokes it (when CWD is $GOPATH/src/github.com/prometheus/prometheus), and 3) is my attempt to fix the problem with 2) by constraining the scope.

More info about each run:

Note: I moved guru to guru-real and replaced guru with this script:

$ cat /home/dieter/go/bin/guru
#!/bin/bash
file="/home/dieter/guru-$(date '+%F_%T')"
echo "guru $@" > $file
env >> $file
guru-real "$@"

this allows me to track the exact environment of the process each time it is run (particularly when the parent is vscode and not my terminal), but it turns out the issue was reproducible perfectly from a terminal, so at this point it's no longer useful, but it does explain why i have mentions of "guru-real" in the pstree output below.

run 1

time /home/dieter/go/bin/guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
    "type": {
        "name": "github.com/prometheus/prometheus/storage.Storage",
        "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
        "kind": "interface"
    },
    "to": [
        {
            "name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
            "kind": "pointer"
        },
        {
            "name": "*github.com/prometheus/prometheus/storage.fanout",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
            "kind": "pointer"
        },
        {
            "name": "*github.com/prometheus/prometheus/storage/remote.Storage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
            "kind": "pointer"
        },
        {
            "name": "*github.com/prometheus/prometheus/tsdb.DB",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
            "kind": "pointer"
        },
        {
            "name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
            "kind": "struct"
        },
        {
            "name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
            "kind": "struct"
        }
    ],
    "from": [
        {
            "name": "github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt.Closer",
            "pos": "/home/dieter/go/src/github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
            "kind": "interface"
        },
        {
            "name": "github.com/prometheus/prometheus/storage.Appendable",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
            "kind": "interface"
        },
        {
            "name": "github.com/prometheus/prometheus/storage.Queryable",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
            "kind": "interface"
        },
        {
            "name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
            "kind": "interface"
        },
        {
            "name": "io.Closer",
            "pos": "/usr/lib/go/src/io/io.go:98:6",
            "kind": "interface"
        }
    ]
}
/home/dieter/go/bin/guru -json implements   95.38s user 26.11s system 733% cpu 16.554 total
pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           `-13*[{guru-real}]
run 2

I didn't let it complete. I killed it after it ran for hours.

pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/hello
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/shape
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/peter-rabbit
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/showdeps
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share2
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/redirect
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/static
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unsafeunmarshaler
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/marshalto
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/populate
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/stringer
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unmarshal
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/addons/kube-proxy
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/gce/manifests
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/juju/prereqs
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/kubemark/pre-existing
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/docs/yaml/kubectl
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-apiserver/app
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-controller-manager/app
  |   |           |   `-11*[{go}]
  |   |           `-35*[{guru-real}]

run 3

time /home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
    "type": {
        "name": "github.com/prometheus/prometheus/storage.Storage",
        "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
        "kind": "interface"
    },
    "to": [
        {
            "name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
            "kind": "pointer"
        },
        {
            "name": "*github.com/prometheus/prometheus/storage.fanout",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
            "kind": "pointer"
        },
        {
            "name": "*github.com/prometheus/prometheus/storage/remote.Storage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
            "kind": "pointer"
        },
        {
            "name": "*github.com/prometheus/prometheus/tsdb.DB",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
            "kind": "pointer"
        },
        {
            "name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
            "kind": "struct"
        },
        {
            "name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
            "kind": "struct"
        }
    ],
    "from": [
        {
            "name": "github.com/prometheus/common/expfmt.Closer",
            "pos": "/home/dieter/go/pkg/mod/github.com/prometheus/common@v0.10.0/expfmt/encode.go:40:6",
            "kind": "interface"
        },
        {
            "name": "github.com/prometheus/prometheus/storage.Appendable",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
            "kind": "interface"
        },
        {
            "name": "github.com/prometheus/prometheus/storage.Queryable",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
            "kind": "interface"
        },
        {
            "name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
            "pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
            "kind": "interface"
        },
        {
            "name": "io.Closer",
            "pos": "/usr/lib/go/src/io/io.go:98:6",
            "kind": "interface"
        }
    ]
}
/home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/...    5478.10s user 886.50s system 246% cpu 42:57.99 total
$ pstree -a -t -l
  |   |   `-guru /home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/... implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json -scope github.com/prometheus/prometheus/... implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/internal/descopts
  |   |           |   `-10*[{go}]
  |   |           |-(go)
  |   |           |   `-{go}
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/runtime/protoimpl
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/util/intstr
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/apis/meta/v1
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/internal/flags
  |   |           |   `-9*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/runtime/schema
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- github.com/prometheus/prometheus/vendor/k8s.io/client-go/plugin/pkg/client
  |   |           |   `-10*[{go}]
  |   |           `-19*[{guru-real}]
stamblerre commented 4 years ago

Thank you for this thorough investigation, but you are right to say that we will not be fixing guru issues. gopls is close to reaching beta, so it is the recommended alternative. If you are interested in using CLI tools, you can also try gopls implementation command, which will tell you both the implementations of a given interface or the interfaces implemented by a concrete type.

adonovan commented 1 month ago

guru was deleted in https://github.com/golang/go/issues/65880; closing as obsolete.