search

golangci / golangci-lint

Fast linters runner for Go
https://golangci-lint.run
GNU General Public License v3.0
15.65k stars 1.39k forks source link

Possible memory leak on macOS running in Docker #5030

Closed feketegy closed 1 month ago

feketegy commented 1 month ago

Welcome

Description of the problem

I can't pinpoint the exact code block that produces 23GB memory usage on macOS running the linter inside Docker in an Alpine Linux 3.20 container, but I don't think this is normal.

image

Version of golangci-lint

```console $ golangci-lint --version # Paste output here golangci-lint has version 1.61.0 built with go1.23.1 from a1d6c560 on 2024-09-09T17:44:42Z ```

Configuration

```console # paste configuration file or CLI flags here --- run: timeout: 1m issues-exit-code: 2 tests: false output: sort-results: true issues: max-same-issues: 0 exclude: - "func \x60(.*)\x60 is unused" linters: disable-all: true enable: - asciicheck - bodyclose - cyclop - dogsled - dupl - errcheck - errname - forcetypeassert - gochecknoinits - gocognit - goimports - gomoddirectives - gosec - gosimple - govet - ineffassign - interfacebloat - misspell - musttag - nestif - noctx - revive - sloglint - staticcheck - stylecheck - tagliatelle - unconvert - unused linters-settings: errcheck: exclude-functions: - (github.com/jackc/pgx/v5.Tx).Rollback dupl: threshold: 200 gosimple: checks: - all - '-S1002' # ignore implicit if conditions - '-S1034' cyclop: max-complexity: 20 skip-tests: true tagliatelle: case: rules: json: goCamel interfacebloat: max: 15 ```

Go environment

```console $ go version && go env # paste output here go version go1.23.1 linux/arm64 GO111MODULE='' GOARCH='arm64' GOBIN='/go/bin' GOCACHE='/root/.cache/go-build' GOENV='/root/.config/go/env' GOEXE='' GOEXPERIMENT='' GOFLAGS='' GOHOSTARCH='arm64' GOHOSTOS='linux' GOINSECURE='' GOMODCACHE='/go/pkg/mod' GONOPROXY='' GONOSUMDB='' GOOS='linux' GOPATH='/go' GOPRIVATE='' GOPROXY='https://proxy.golang.org,direct' GOROOT='/usr/local/go' GOSUMDB='sum.golang.org' GOTMPDIR='' GOTOOLCHAIN='local' GOTOOLDIR='/usr/local/go/pkg/tool/linux_arm64' GOVCS='' GOVERSION='go1.23.1' GODEBUG='' GOTELEMETRY='local' GOTELEMETRYDIR='/root/.config/go/telemetry' GCCGO='gccgo' GOARM64='v8.0' AR='ar' CC='gcc' CXX='g++' CGO_ENABLED='0' GOMOD='/srv/api/go.mod' GOWORK='' CGO_CFLAGS='-O2 -g' CGO_CPPFLAGS='' CGO_CXXFLAGS='-O2 -g' CGO_FFLAGS='-O2 -g' CGO_LDFLAGS='-O2 -g' PKG_CONFIG='pkg-config' GOGCCFLAGS='-fPIC -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2615380691=/tmp/go-build -gno-record-gcc-switches' ```

Verbose output of running

```console $ golangci-lint cache clean $ golangci-lint run -v # paste output here INFO golangci-lint has version 1.61.0 built with go1.23.1 from a1d6c560 on 2024-09-09T17:44:42Z INFO [config_reader] Config search paths: [./ /srv/api /srv / /root] INFO [config_reader] Used config file .golangci.yaml INFO [lintersdb] Active 28 linters: [asciicheck bodyclose cyclop dogsled dupl errcheck errname forcetypeassert gochecknoinits gocognit goimports gomoddirectives gosec gosimple govet ineffassign interfacebloat misspell musttag nestif noctx revive sloglint staticcheck stylecheck tagliatelle unconvert unused] INFO [loader] Go packages loading at mode 575 (compiled_files|deps|exports_file|imports|types_sizes|files|name) took 1.029456625s INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 4.380583ms INFO [linters_context/goanalysis] analyzers took 10.005694927s with top 10 stages: buildir: 6.58659867s, the_only_name: 1.035208416s, inspect: 652.832118ms, ctrlflow: 222.925792ms, fact_purity: 197.85613ms, printf: 194.525955ms, fact_deprecated: 190.31208ms, nilness: 144.944672ms, SA5012: 107.28158ms, musttag: 106.767582ms INFO [runner] Issues before processing: 135, after processing: 0 INFO [runner] Processors filtering stat (in/out): filename_unadjuster: 135/135, invalid_issue: 135/135, skip_files: 135/135, identifier_marker: 135/135, exclude: 135/128, path_prettifier: 135/135, autogenerated_exclude: 135/135, skip_dirs: 135/135, exclude-rules: 128/0, cgo: 135/135 INFO [runner] processing took 11.15133ms with stages: autogenerated_exclude: 4.32375ms, path_prettifier: 2.665292ms, exclude-rules: 1.770584ms, identifier_marker: 1.438583ms, filename_unadjuster: 387.708µs, skip_dirs: 353.375µs, exclude: 161.042µs, invalid_issue: 34.083µs, cgo: 11.917µs, sort_results: 1.125µs, skip_files: 1.084µs, nolint: 834ns, fixer: 667ns, max_same_issues: 332ns, diff: 208ns, max_from_linter: 208ns, uniq_by_line: 167ns, path_shortener: 124ns, severity-rules: 83ns, source_code: 82ns, max_per_file_from_linter: 82ns, path_prefixer: 0s INFO [runner] linters took 2.257625584s with stages: goanalysis_metalinter: 2.246390126s INFO File cache stats: 46 entries of total size 62.7KiB INFO Memory: 34 samples, avg is 248.0MB, max is 501.8MB INFO Execution took 3.295576793s ```

A minimal reproducible example or link to a public repository

```go // add your code here ```

Validation

Supporter

boring-cyborg[bot] commented 1 month ago

Hey, thank you for opening your first Issue ! 🙂 If you would like to contribute we have a guide for contributors.

benoitgravitee commented 1 month ago

+1 Got stuck with never ending raise of memory. Killed it, but realised later I got a dangling process ending up filling the swap up to 32GB! Maybe an issue with the go version, I'm using 1.23, is it compatible ?

ldez commented 1 month ago

If you are facing the same problem, the best way to contribute is to provide the following information:

A code example or link to a public repository.

golangci-lint --version
cat .golangci.yml
go version && go env
golangci-lint cache clean
golangci-lint run -v

You can also read the following issues: #5012, #4989, #4933, #4909, #4908, #4932 Related to #4837

feketegy commented 1 month ago

@ldez Unfortunately I can't post a code sample because I cannot pinpoint exactly where the problem is happening in my code base. The rest of the requirements for opening the issue is all there.

ldez commented 1 month ago

@feketegy my message was mainly for @benoitgravitee.

Can you provide the information about the image you are using?

feketegy commented 1 month ago

Can you provide the information about the image you are using?

Sure, it's the basic arm64v8/golang:1.23.1-alpine3.20 Docker image. This is my Dockerfile, I removed some of the project code commands for obvious reasons.

I hope this helps.

FROM arm64v8/golang:1.23.1-alpine3.20 AS base

RUN apk update \
  && apk add make \
  && apk add git \
  && apk add mercurial \
  && apk add bash \
  && apk add curl \
  && apk add musl-dev \
  && apk add vips-dev \
  && apk add gcc

FROM base AS api

ENV GOPATH /go
ENV GOBIN /go/bin
ENV PATH "$PATH:$GOPATH/bin"

RUN curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.61.0 \
  && go install github.com/matryer/moq@latest \
  && go install github.com/cespare/reflex@latest
Mexx77 commented 1 month ago

we saw usage of +30GB memory and a stuck process when enable gosimple linter in Go 1.23.1. With Go 1.22.5 we didn't have that issue

ldez commented 1 month ago

@Mexx77 If you are facing the same problem, the best way to contribute is to provide the following information:

A code example or link to a public repository.

golangci-lint --version
cat .golangci.yml
go version && go env
golangci-lint cache clean
golangci-lint run -v

You can read the following documentation: https://golangci-lint.run/welcome/faq/#which-go-versions-are-supported You can also read the following issues: #5012, #4989, #4933, #4909, #4908, #4932 Related to #4837

ldez commented 1 month ago

I don't have a mac or arm64 computer, so I adapted the Dockerfile for Linux/amd64 and run it on golangci-lint code.

I see no unexpected memory usage.

I think to different context that could lead to extra memory consumption:

```dockerfile FROM golang:1.23.1-alpine3.20 AS base RUN apk update \ && apk add make \ && apk add git \ && apk add mercurial \ && apk add bash \ && apk add curl \ && apk add musl-dev \ && apk add vips-dev \ && apk add gcc FROM base AS api ENV GOPATH=/go ENV GOBIN=/go/bin ENV PATH="$PATH:$GOPATH/bin" RUN curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.61.0 # Set all directories as safe RUN git config --global --add safe.directory '*' ``` ```bash docker build -t sandbox:temp . ``` ```console $ docker run --rm -v $(pwd):/app -w /app -it sandbox:temp sh /app # go mod tidy go: downloading github.com/spf13/pflag v1.0.5 ... go: downloading github.com/go-task/slim-sprig/v3 v3.0.0 /app # golangci-lint version golangci-lint has version 1.61.0 built with go1.23.1 from a1d6c560 on 2024-09-09T17:44:42Z /app # golangci-lint run /app # ```