search

snail2sky / coredns_mysql_extend

connect mysql plugin of coredns
Apache License 2.0
9 stars 2 forks source link

fatal error: concurrent map read and map write #4

Closed mm3906078 closed 2 months ago

mm3906078 commented 3 months ago

Hi there, I got this error on my coredns and exited with 2 returned.

[ERROR] plugin/errors: 2 X.X.X.X.in-addr.arpa. PTR: plugin/mysql: no next plugin found                                              
fatal error: concurrent map read and map write                                                                                      

goroutine 141310 [running]:{0x0, 0x0}, {0x22d2a60, 0x3}, {0x0, ...}, ...github.com/snail2sky/coredns_mysql_extend.(*Mysql).degradeQu
ery(0x312e322e302e3031?, {0x0, 0x0, {0x0, 0x0},                                                                                     
})                                                                                                                                  
        /go/pkg/mod/github.com/snail2sky/coredns_mysql_extend@v1.1.5/utils.go:63 +0x4a, {0x297f1c8, 0xc000e6ade0}, {0x2987dd8, 0xc00
0b79c00}, 0xc000d12240github.com/snail2sky/coredns_mysql_extend.(*Mysql).ServeDNS(0xc0006df830                                      
)                                                                                                                                   
        /go/pkg/mod/github.com/snail2sky/coredns_mysql_extend@v1.1.5/mysql.go:131 +0x1865{0x22d3be2?, 0x29881f8?}, {0x2966bf0, 0xc00
06df830}, {0x297f1c8, 0xc000e6ade0github.com/coredns/coredns/plugin.NextOrFailure(                                                  
}, {0x2987dd8, 0xc000b79c00}, 0xc000d12240)     /go/coredns/plugin/plugin.go:80                                                     
 +0x289                                                                                                                             
github.com/coredns/coredns/plugin/cache.(*Cache).doRefresh(...) /go/coredns/plugin/cache/handler.go:107                             
github.com/coredns/coredns/plugin/cache.(*Cache).ServeDNS(0xc000150c00,                                                             
{0x297f1c8, 0xc000e6ade0}, {0x29881f8, 0xc001573480}, 0xc000d121b0)                                                                 
        /go/coredns/plugin/cache/handler.go:41 +0xd6fgithub.com/coredns/coredns/plugin.NextOrFailure({0x22d299d?,                   
0x1?}, {0x2965fe8, 0xc000150c00}, {0x297f1c8, 0xc000e6ade0}, {0x29881f8, 0xc001573480}, 0xc000d121b0)                               
        /go/coredns/plugin/plugin.go:80 +0x289github.com/coredns/coredns/plugin/log.Logger.ServeDNS({{0x2965fe8, 0xc000150c00}, {0xc
00033b1d0, 0x1,                                                                                                                     
0x1}, {}}, {0x297f1c8, 0xc000e6ade0}, {0x2988250, 0xc001573440}, ...)                                                               
        /go/coredns/plugin/log/log.go:36 +0x2bf}, {0x2988250, 0xc001573440}, 0xc000d121b0}, {0x29666f0, 0xc0006df950}, {0x297f1c8, 0
xc000e6ade0github.com/coredns/coredns/plugin.NextOrFailure({0x22d4ecd?, 0x3c03720?                                                  
)                                                                                                                                   
        /go/coredns/plugin/plugin.go:80 +0x289(0xc00033b170, {0x297f1c8?, 0xc000e6ade0?}, {0x2988250, 0xc001573440github.com/coredns
/coredns/plugin/errors.(*errorHandler).ServeDNS                                                                                     
}, 0xc000d121b0)                                                                                                                    
        /go/coredns/plugin/errors/errors.go:84 +0x7b0x297f1c8, 0xc000e6ade0}, {0x2988250, 0xc001573440{0x22db8de?, 0x0?}, {0x2966240
, 0xc00033b170}, {github.com/coredns/coredns/plugin.NextOrFailure(                                                                  
}, 0xc000d121b0)                                                                                                                    
        /go/coredns/plugin/plugin.go:80 +0x289github.com/coredns/coredns/plugin/metrics.(*Metrics).ServeDNS(0xc0005e9560, {0x297f1c8
, 0xc000e6ade0}, {0x2988618, 0xc000d08810}, 0xc000d121b0)       /go/coredns/plugin/metrics/handler.go                               
:27 +0x245github.com/coredns/coredns/core/dnsserver.(*Server).ServeDNS(0xc00052e680, {0x297f1c8, 0xc000e6ade0}                      
, {0x2988618, 0xc000d08810}, 0xc000d121b0)      /go/coredns/core/dnsserver/server.go:                                               
316 +0xc1agithub.com/coredns/coredns/core/dnsserver.(*Server).ServePacket.func1({                                                   
0x2989c18, 0xc000d0d080}, 0xc000d121b0) /go/coredns/core/dnsserver/server.go                                                        
:178 +0x8e(0xc00033c000?, {0x2989c18?, github.com/miekg/dns.HandlerFunc.ServeDNS                                                    
0xc000d0d080?}, 0xc000d121b0?)  /go/pkg/mod/github.com/miekg/dns@v1.1.55/server.go                                                  
:37 +0x29github.com/miekg/dns.(*Server).serveDNS(0xc0006ad0e0, {                                                                    
0xc00033c000, 0x39, 0x200}, 0xc000d0d080)       /go/pkg/mod/github.com/miekg/dns@v1.1.55/server.go:                                 
659 +0x42a}, {0x29862e0, 0xc0004ac980}, 0xc001d1b760, , 0xc00052ada0, {0xc00033c000, 0x39, 0x200github.com/miekg/dns.(*Server).serve
UDPPacket(0xc0006ad0e0                                                                                                              
{0x0, 0x0})                                                                                                                         
        /go/pkg/mod/github.com/miekg/dns@v1.1.55/server.go:603 +0x1a5                                                               
created by github.com/miekg/dns.(*Server).serveUDP in goroutine 87                                                                          /go/pkg/mod/github.com/miekg/dns@v1.1.55/server.go:533 +0x405                                                               

goroutine 1 [semacquire, 11 minutes]:                                                                                               sync.runtime_Semacquire(0xc000708ca0?)                                                                                              
        /usr/local/go/src/runtime/sema.go:62 +0x25sync.(*WaitGroup).Wait(0x296e9a8                                                  ?)                                                                                                                                  
        /usr/local/go/src/sync/waitgroup.go:116 +0x48github.com/coredns/caddy.(*Instance).Wait                                      (...)                                                                                                                               
        /go/pkg/mod/github.com/coredns/caddy@v1.1.1/caddy.go:428                                                                    
github.com/coredns/coredns/coremain.Run()                                                                                           
        /go/coredns/coremain/run.go:76 +0x21amain.main(                                                                             
)                                                                                                                                           /go/coredns/coredns.go:12 +0xf                                                                                              
goroutine 3                                                                                                                         
 [select]:                                                                                                                          go.opencensus.io/stats/view.(*worker).start(0xc000593580)       /go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:          
292 +0x9f                                                                                                                           
created by go.opencensus.io/stats/view.init.0 in goroutine 1                                                                        
/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x8d                                                                  goroutine 103 [chan receive                                                                                                         
, 11 minutes]:                                                                                                                      github.com/coredns/caddy.trapSignalsCrossPlatform.func1()                                                                           
        /go/pkg/mod/github.com/coredns/caddy@v1.1.1/sigtrap.go:43 +0x8e                                                             
created by github.com/coredns/caddy.trapSignalsCrossPlatform in goroutine 1     /go/pkg/mod/github.com/coredns/caddy@v1.1.1/sigtrap.go:38                                                                                                                               
 +0x1a                                                                                                                              
goroutine 104 [chan receive,                                                                                                        11 minutes]:                                                                                                                        
github.com/coredns/caddy.trapSignalsPosix.func1()                                                                                   
        /go/pkg/mod/github.com/coredns/caddy@v1.1.1/sigtrap_posix.go:32 +0x105created by                                            github.com/coredns/caddy.trapSignalsPosix in goroutine 1                                                                            
        /go/pkg/mod/github.com/coredns/caddy@v1.1.1/sigtrap_posix.go:28 +0x1a                                                       
                                                                                                                                    goroutine 43 [syscall, 11 minutes]:                                                                                                 
os/signal.signal_recv()                                                                                                             
        /usr/local/go/src/runtime/sigqueue.go:152 +0x29                                                                             
os/signal.loop()        /usr/local/go/src/os/signal/signal_unix.go:23 +0x13                                                                                                                                                                                             
created by os/signal.Notify.func1.1 in goroutine 103                                                                                
        /usr/local/go/src/os/signal/signal.go:151 +0x1f                                                                             

goroutine 105 [IO wait, 10 minutes]:
internal/poll.runtime_pollWait(0x7f2fed73bda8, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85
internal/poll.(*pollDesc).wait(0x7?, 0x3fe?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:
89
internal/poll.(*FD).Accept(0xc00052e700)
        /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac
net.(*netFD).accept(0xc00052e700) 
        /usr/local/go/src/net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc00060d9e0) /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e

also this is my config:

in-addr.arpa.:53 {
    log
    errors
    mysql {
        dsn USER:PASS@tcp(mysql:3306)/coredns
    }
    prometheus :9153
    cache {
        success 4096
        denial 1024
        servfail 5m
    }
}
snail2sky commented 3 months ago

1.5/utils.go

Sorry, I just saw this issue. I will deal with it right away.

snail2sky commented 3 months ago

in-addr.arpa.:53 { log errors mysql { dsn USER:PASS@tcp(mysql:3306)/coredns } prometheus :9153 cache { success 4096 denial 1024 servfail 5m } }

Can you provide a reproduction method? There is currently no reproduction success. Is greater concurrency required to reproduce?