search

kevinjqiu / coredns-dockerdiscovery

Docker Discovery Plugin for CoreDNS
Apache License 2.0
62 stars 29 forks source link

Protect maps with sync.RWMutex #22

Closed schoentoon closed 2 years ago

schoentoon commented 2 years ago

After running this plugin for a couple of hours, I noticed it had crashed coredns with the following stacktrace.

fatal error: concurrent map writes

goroutine 43989 [running]:
runtime.throw({0xd4b6fc?, 0x40?})
      /usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc0006b7c60 sp=0xc0006b7c30 pc=0x4383f1
runtime.mapassign_faststr(0xc41fc0, 0xc000473aa0, {0xc0006f6a00, 0x40})
      /usr/local/go/src/runtime/map_faststr.go:295 +0x38b fp=0xc0006b7cc8 sp=0xc0006b7c60 pc=0x4135cb
github.com/kevinjqiu/coredns-dockerdiscovery.DockerDiscovery.updateContainerInfo({{0x0, 0x0}, {0xc0003e1620, 0x1b}, {0xc000238040, 0x3, 0x4}, 0xc000ae86c0, 0xc000473aa0, 0x0}, ...)
      /go/pkg/mod/github.com/kevinjqiu/coredns-dockerdiscovery@v0.0.0-20220324043809-78fca25a3f08/dockerdiscovery.go:168 +0x3e5 fp=0xc0006b7dd8 sp=0xc0006b7cc8 pc=0xb73905
github.com/kevinjqiu/coredns-dockerdiscovery.DockerDiscovery.start.func1(0xc000968f80)
      /go/pkg/mod/github.com/kevinjqiu/coredns-dockerdiscovery@v0.0.0-20220324043809-78fca25a3f08/dockerdiscovery.go:247 +0x858 fp=0xc0006b7fc8 sp=0xc0006b7dd8 pc=0xb74958
github.com/kevinjqiu/coredns-dockerdiscovery.DockerDiscovery.start.func2()
      /go/pkg/mod/github.com/kevinjqiu/coredns-dockerdiscovery@v0.0.0-20220324043809-78fca25a3f08/dockerdiscovery.go:262 +0x2a fp=0xc0006b7fe0 sp=0xc0006b7fc8 pc=0xb740ca
runtime.goexit()
      /usr/local/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc0006b7fe8 sp=0xc0006b7fe0 pc=0x46b201
created by github.com/kevinjqiu/coredns-dockerdiscovery.DockerDiscovery.start
      /go/pkg/mod/github.com/kevinjqiu/coredns-dockerdiscovery@v0.0.0-20220324043809-78fca25a3f08/dockerdiscovery.go:219 +0x116

Basically a classic race condition on a map, so throwing Lock() calls around it should be enough.

kevinjqiu commented 2 years ago

Nice catch!