topolvm / pvc-autoresizer

Auto-resize PersistentVolumeClaim objects based on Prometheus metrics
Apache License 2.0
249 stars 29 forks source link

fatal error: concurrent map writes #266

Closed Mihirkumarsingh closed 4 months ago

Mihirkumarsingh commented 4 months ago

Describe the bug Fatal error: concurrent map writes

Environments

To Reproduce Steps to reproduce the behavior:

Expected behavior i would expect controller to run fine

Additional context

{"level":"info","ts":"2024-06-28T16:05:20Z","logger":"setup","msg":"starting manager"}

{"level":"info","ts":"2024-06-28T16:05:20Z","logger":"controller-runtime.metrics","msg":"Starting metrics server"}

{"level":"info","ts":"2024-06-28T16:05:20Z","msg":"starting server","kind":"health probe","addr":":8081"}

{"level":"info","ts":"2024-06-28T16:05:20Z","logger":"controller-runtime.metrics","msg":"Serving metrics server","bindAddress":":8080","secure":false}

{"level":"info","ts":"2024-06-28T16:05:20Z","logger":"controller-runtime.webhook","msg":"Starting webhook server"}

{"level":"info","ts":"2024-06-28T16:05:20Z","logger":"controller-runtime.certwatcher","msg":"Updated current TLS certificate"}

{"level":"info","ts":"2024-06-28T16:05:20Z","logger":"controller-runtime.webhook","msg":"Serving webhook server","host":"","port":9443}

{"level":"info","ts":"2024-06-28T16:05:20Z","logger":"controller-runtime.certwatcher","msg":"Starting certificate watcher"}

I0628 16:05:20.535385 1 leaderelection.go:250] attempting to acquire leader lease pep-system/49e22f61.topolvm.io...

I0628 16:05:38.710105 1 leaderelection.go:260] successfully acquired lease pep-system/49e22f61.topolvm.io

fatal error: concurrent map writes

goroutine 932 [running]:

github.com/topolvm/pvc-autoresizer/internal/runners.getPVCUsage(0x0?, {0xc00070c380, 0x33}, 0xc000271ef0, {0x1bc23c0, 0xc0021b0000})

/workspace/internal/runners/k8s_metrics_api_client.go:87 +0x79e

github.com/topolvm/pvc-autoresizer/internal/runners.(*k8sMetricsApiClient).GetMetrics.func1()

net.(*netFD).accept(0xc00048c380)

/usr/local/go/src/net/fd_unix.go:172 +0x29

net.(*TCPListener).accept(0xc0003f2580)

/usr/local/go/src/net/tcpsock_posix.go:159 +0x1e

net.(*TCPListener).Accept(0xc0003f2580)

/usr/local/go/src/net/tcpsock.go:327 +0x30

crypto/tls.(*listener).Accept(0xc0004a06a8)

/usr/local/go/src/crypto/tls/tls.go:66 +0x27

net/http.(*Server).Serve(0xc00032c000, {0x1bb8160, 0xc0004a06a8})

/usr/local/go/src/net/http/server.go:3255 +0x33e

sigs.k8s.io/controller-runtime/pkg/webhook.(*DefaultServer).Start(0xc000258000, {0x1bc23c0, 0xc00011def0})

[/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.5/pkg/webhook/server.go:263](mailto:/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.5/pkg/webhook/server.go:263) +0x885

sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc0001e7660)

[/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.5/pkg/manager/runnable_group.go:223](mailto:/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.5/pkg/manager/runnable_group.go:223) +0xc8

created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 86

[/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.5/pkg/manager/runnable_group.go:207](mailto:/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.5/pkg/manager/runnable_group.go:207) +0x19d
daichimukai commented 4 months ago

Hi @Mihirkumarsingh,

Thank you for your bug report! We've addressed the issue in #268. The next release, planned for August, will include this fix.