search

golangci / golangci-lint

Fast linters runner for Go
https://golangci-lint.run
GNU General Public License v3.0
15.65k stars 1.39k forks source link

Performance issues with `musttag` linter #3910

Closed phisco closed 1 year ago

phisco commented 1 year ago

Welcome

Description of the problem

On https://github.com/upbound/provider-aws, and other similar projects, we saw some major performance degradation upgrading from an older version to v1.50.x, we tried doing some investigation back then and couldn't find any obvious reason, we ended up bumping the specs of our runners, which was enough to solve the issue. We now tried looking into into it further, given that we also want users to run it locally before opening a PR, so we did some more analysis and discovered most of the time was lost on the musttag linter this time as you can see from the syscall graph below: image Just disabling it we are now back at a much healthier resource consumption and graph, we went from crashing due to the default 10m timeout with musttag enabled to having it successfully run in around 2 minutes after having disabled it. Taking care to have cleaned up the cache before every run with golangci-lint cache clean. Here the graph with musttag disabled: image

Tests run on MacBook Pro 14" M1 Pro 32GB, but also on a linux box with similar results.

We couldn't find any other similar issue, we are indeed running it successfully on https://github.com/crossplane/crossplane/ , so maybe there is something pathological with our projects that triggers such a behaviour, but could be worth investigating nonetheless.

Version of golangci-lint

```console $ golangci-lint --version golangci-lint has version 1.53.2 built with go1.20.4 from 59a7aaf on 2023-06-03T10:44:21Z ```

Configuration file

see https://github.com/upbound/provider-aws/blob/main/.golangci.yml ```console $ cat .golangci.yml run: deadline: 10m skip-files: - "zz_\\..+\\.go$" output: # colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number" format: colored-line-number linters-settings: errcheck: # report about not checking of errors in type assetions: `a := b.(MyStruct)`; # default is false: such cases aren't reported by default. check-type-assertions: false # report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`; # default is false: such cases aren't reported by default. check-blank: false # [deprecated] comma-separated list of pairs of the form pkg:regex # the regex is used to ignore names within pkg. (default "fmt:.*"). # see https://github.com/kisielk/errcheck#the-deprecated-method for details ignore: fmt:.*,io/ioutil:^Read.* govet: # report about shadowed variables check-shadowing: false revive: # confidence for issues, default is 0.8 confidence: 0.8 gofmt: # simplify code: gofmt with `-s` option, true by default simplify: true goimports: # put imports beginning with prefix after 3rd-party packages; # it's a comma-separated list of prefixes local-prefixes: github.com/upbound/provider-aws gocyclo: # minimal code complexity to report, 30 by default (but we recommend 10-20) min-complexity: 10 maligned: # print struct with more effective memory layout or not, false by default suggest-new: true dupl: # tokens count to trigger issue, 150 by default threshold: 100 goconst: # minimal length of string constant, 3 by default min-len: 3 # minimal occurrences count to trigger, 3 by default min-occurrences: 5 lll: # tab width in spaces. Default to 1. tab-width: 1 unused: # treat code as a program (not a library) and report unused exported identifiers; default is false. # XXX: if you enable this setting, unused will report a lot of false-positives in text editors: # if it's called for subdir of a project it can't find funcs usages. All text editor integrations # with golangci-lint call it on a directory with the changed file. check-exported: false unparam: # Inspect exported functions, default is false. Set to true if no external program/library imports your code. # XXX: if you enable this setting, unparam will report a lot of false-positives in text editors: # if it's called for subdir of a project it can't find external interfaces. All text editor integrations # with golangci-lint call it on a directory with the changed file. check-exported: false nakedret: # make an issue if func has more lines of code than this setting and it has naked returns; default is 30 max-func-lines: 30 prealloc: # XXX: we don't recommend using this linter before doing performance profiling. # For most programs usage of prealloc will be a premature optimization. # Report preallocation suggestions only on simple loops that have no returns/breaks/continues/gotos in them. # True by default. simple: true range-loops: true # Report preallocation suggestions on range loops, true by default for-loops: false # Report preallocation suggestions on for loops, false by default gocritic: # Enable multiple checks by tags, run `GL_DEBUG=gocritic golangci-lint` run to see all tags and checks. # Empty list by default. See https://github.com/go-critic/go-critic#usage -> section "Tags". enabled-tags: - performance settings: # settings passed to gocritic captLocal: # must be valid enabled check name paramsOnly: true rangeValCopy: sizeThreshold: 32 linters: enable: - megacheck - govet - gocyclo - interfacer - goconst - goimports - gofmt # We enable this as well as goimports for its simplify mode. - prealloc - revive - unconvert - misspell - nakedret presets: - bugs - unused fast: false issues: # Excluding configuration per-path and per-linter exclude-rules: # Exclude some linters from running on tests files. - path: _test(ing)?\.go linters: - gocyclo - errcheck - dupl - gosec - scopelint - unparam # Ease some gocritic warnings on test files. - path: _test\.go text: "(unnamedResult|exitAfterDefer)" linters: - gocritic # These are performance optimisations rather than style issues per se. # They warn when function arguments or range values copy a lot of memory # rather than using a pointer. - text: "(hugeParam|rangeValCopy):" linters: - gocritic # This "TestMain should call os.Exit to set exit code" warning is not clever # enough to notice that we call a helper method that calls os.Exit. - text: "SA3000:" linters: - staticcheck - text: "k8s.io/api/core/v1" linters: - goimports # This is a "potential hardcoded credentials" warning. It's triggered by # any variable with 'secret' in the same, and thus hits a lot of false # positives in Kubernetes land where a Secret is an object type. - text: "G101:" linters: - gosec - gas # This is an 'errors unhandled' warning that duplicates errcheck. - text: "G104:" linters: - gosec - gas # Independently from option `exclude` we use default exclude patterns, # it can be disabled by this option. To list all # excluded by default patterns execute `golangci-lint run --help`. # Default value for this option is true. exclude-use-default: false # Show only new issues: if there are unstaged changes or untracked files, # only those changes are analyzed, else only changes in HEAD~ are analyzed. # It's a super-useful option for integration of golangci-lint into existing # large codebase. It's not practical to fix all existing issues at the moment # of integration: much better don't allow issues in new code. # Default is false. new: false # Maximum issues count per one linter. Set to 0 to disable. Default is 50. max-per-linter: 0 # Maximum count of issues with the same text. Set to 0 to disable. Default is 3. max-same-issues: 0 ```

Go environment

```console $ go version && go env GO111MODULE="on" GOARCH="arm64" GOBIN="" GOCACHE="/Users/phisco/Library/Caches/go-build" GOENV="/Users/phisco/Library/Application Support/go/env" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="arm64" GOHOSTOS="darwin" GOINSECURE="" GOMODCACHE="/Users/phisco/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/phisco/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/opt/homebrew/Cellar/go/1.20.5/libexec" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/opt/homebrew/Cellar/go/1.20.5/libexec/pkg/tool/darwin_arm64" GOVCS="" GOVERSION="go1.20.5" GCCGO="gccgo" AR="ar" CC="cc" CXX="c++" CGO_ENABLED="1" GOMOD="/Users/phisco/github.com/upbound/provider-aws/go.mod" GOWORK="" CGO_CFLAGS="-O2 -g" CGO_CPPFLAGS="" CGO_CXXFLAGS="-O2 -g" CGO_FFLAGS="-O2 -g" CGO_LDFLAGS="-O2 -g" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/w9/kmd32ffd0hg7lq0rt7qk7cjw0000gn/T/go-build4095682310=/tmp/go-build -gno-record-gcc-switches -fno-common" ```

Verbose output of running

```console $ golangci-lint cache clean $ golangci-lint run -v INFO [config_reader] Config search paths: [./ /Users/phisco/github.com/upbound/provider-aws /Users/phisco/github.com/upbound /Users/phisco/github.com /Users/phisco /Users /] INFO [config_reader] Used config file .golangci.yml INFO [lintersdb] Active 41 linters: [asasalint asciicheck bidichk bodyclose contextcheck durationcheck errcheck errchkjson errorlint exhaustive exportloopref gocheckcompilerdirectives goconst gocritic gocyclo gofmt goimports gosec gosimple gosmopolitan govet ineffassign interfacer loggercheck makezero misspell musttag nakedret nilerr noctx prealloc reassign revive rowserrcheck sqlclosecheck staticcheck typecheck unconvert unparam unused zerologlint] INFO [lintersdb] Active presets: [bugs unused] INFO [loader] Go packages loading at mode 575 (compiled_files|imports|name|types_sizes|deps|exports_file|files) took 1.963875792s WARN [runner] The linter 'interfacer' is deprecated (since v1.38.0) due to: The repository of the linter has been archived by the owner. INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 157.242ms INFO Memory: 5600 samples, avg is 5217.4MB, max is 6585.2MB INFO Execution took 10m0.091753375s INFO [linters_context/goanalysis] analyzers took 3h42m3.768296194s with top 10 stages: musttag: 1h43m47.415679784s, gocritic: 31m37.846701362s, buildir: 17m1.368766982s, buildssa: 15m53.661087364s, bidichk: 3m19.573175141s, unconvert: 2m20.184143887s, goimports: 2m19.56888887s, the_only_name: 2m0.045592431s, interfacer: 1m59.08095479s, unparam: 1m26.580631903s INFO [runner] Issues before processing: 4471, after processing: 0 INFO [runner] Processors filtering stat (out/in): cgo: 4471/4471, filename_unadjuster: 4471/4471, skip_files: 4471/4471, exclude-rules: 8/177, path_prettifier: 4471/4471, autogenerated_exclude: 177/4471, identifier_marker: 177/177, nolint: 0/8, skip_dirs: 4471/4471, exclude: 177/177 INFO [runner] processing took 183.001084ms with stages: autogenerated_exclude: 111.967083ms, path_prettifier: 52.813999ms, skip_dirs: 12.9235ms, identifier_marker: 2.133792ms, nolint: 1.709166ms, exclude-rules: 519.75µs, skip_files: 418.292µs, cgo: 325.376µs, filename_unadjuster: 186.584µs, max_same_issues: 959ns, fixer: 459ns, uniq_by_line: 375ns, source_code: 375ns, exclude: 251ns, sort_results: 208ns, path_prefixer: 208ns, severity-rules: 208ns, diff: 207ns, path_shortener: 125ns, max_from_linter: 84ns, max_per_file_from_linter: 83ns INFO [runner] linters took 10m58.974005917s with stages: goanalysis_metalinter: 10m58.790906583s INFO File cache stats: 3122 entries of total size 28.7MiB ERRO Timeout exceeded: try increasing it by passing --timeout option ```

Code example or link to a public repository

https://github.com/upbound/provider-aws/
boring-cyborg[bot] commented 1 year ago

Hey, thank you for opening your first Issue ! 🙂 If you would like to contribute we have a guide for contributors.

ldez commented 1 year ago

Hello,

$ golangci-lint version
golangci-lint has version 1.53.2 built with go1.20.4 from 59a7aaf7 on 2023-06-03T15:04:31Z
$ git clone git@github.com:upbound/provider-aws.git
Cloning into 'provider-aws'...

$ cd provider-aws      
$ golangci-lint run                                
WARN [runner] The linter 'interfacer' is deprecated (since v1.38.0) due to: The repository of the linter has been archived by the owner.  
7m35s $
$ golangci-lint run    
WARN [runner] The linter 'interfacer' is deprecated (since v1.38.0) due to: The repository of the linter has been archived by the owner.
2s $

Note: the 2nd call to golangci-lint run uses only the cache, so the problem is not related to the use of the cache.

If I disable musttag:

$ golangci-lint run        
WARN [runner] The linter 'interfacer' is deprecated (since v1.38.0) due to: The repository of the linter has been archived by the owner.  
1m37s $

Also, your graph shows that it's related to musttag only.

You checked the following box:

Can you provide the output and the duration of standalone binary of musttag?

ldez commented 1 year ago

The problem is related to the go list calls inside musttag https://github.com/tmzane/musttag/blob/main/utils.go

I created a custom version to display the calls: ```console $ ./golangci-lint run --no-config --disable-all -Emusttag go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all go list -m -f={{.Dir}} go list -f={{if and (not .Standard) .Module.Main}}{{.ImportPath}}{{end}} all ... ```
ldez commented 1 year ago

So it's a problem with musttag, it's a side effect of a bug fix.

phisco commented 1 year ago

thank you @ldez for the super quick reply! yes, this was the same conclusion I came to, should I open an issue on the musttag linter instead? looks like the repo was moved from github.com/junk1tm/musttag to github.com/tmzane/musttag, right?

here the output running it via govet, not much:

$ time go vet -vettool=$(which musttag) ./...
go vet -vettool=$(which musttag) ./...  2672.40s user 1363.83s system 749% cpu 8:58.46 total

and here the output running it directly with debug logs enabled:

time musttag -debug fpstv ./...                                                                                        130 ↵
15:54:10.850971 load [./...]
15:54:13.490103 building graph of analysis passes

... I'm still waiting, I'll update this once it finishes ...

I was looking at my clone and thought I did pull the latest changes but I did not 😅 : I also noticed golangci-lint is using v0.5.0 of musttag, while the latest seems to be 0.7.0, but it doesn't look like that should change the code running the go list🤔

ldez commented 1 year ago

I also noticed golangci-lint is using v0.5.0 of musttag

golangci-lint used v0.7.0. https://github.com/golangci/golangci-lint/blob/d147d8bf4f8530fb0384614a4ebcc1defe0796bc/go.mod#L118

phisco commented 1 year ago

yes, sorry, I edited the message above, I was looking at an old version of the repo and forgot to pull

ldez commented 1 year ago

just for sharing: The reason behind the issue, it's that your project has a lot of small packages, so a lot of Run and a lot of go list. It's not related LOC but the number of packages.

phisco commented 1 year ago

Makes sense, thanks! I'll let the maintainers know that reducing the number of packages could be another option.

Btw, I'll take the chance to thank you for this awesome project, I've been using it for a few years now and I couldn't work without it 🙏

ldez commented 1 year ago

I created a fix, the output with your repo and my changes:

$ ./golangci-lint run                                                  
1m53s $
phisco commented 1 year ago

@ldez feel free to close this issue or to keep it around until the musttag dependency is bumped 👏 thank you so much again!

Edit to add the missing context: A new version was released with the fix https://github.com/tmzane/musttag/issues/55

ldez commented 1 year ago

I will close it when PR #3924 will be merged.