tidwall / evio

Fast event-loop networking for Go
MIT License
5.88k stars 491 forks source link

numLoops > 1 will make udp server slower #73

Closed tianlin closed 1 year ago

tianlin commented 2 years ago

Hi: I found when set numLoops > 1, udp server with evio will be slower than numLoops = 1. Use profile, I found there is too much syscall in loopUDPRead(s, l, i, fd).
I add a debug code fmt.Println("-- loop udp read --", l.idx) before loopUDPRead, then send just one udp msg to it, there are numLoops's print in stdout. I change the LoadBalance to AtLeastConnections / RoundRobin, there is also more call to loopUDPRead when numLoops > 1.

cheng-zhongliang commented 1 year ago

Hi: I found when set numLoops > 1, udp server with evio will be slower than numLoops = 1. Use profile, I found there is too much syscall in loopUDPRead(s, l, i, fd). I add a debug code fmt.Println("-- loop udp read --", l.idx) before loopUDPRead, then send just one udp msg to it, there are numLoops's print in stdout. I change the LoadBalance to AtLeastConnections / RoundRobin, there is also more call to loopUDPRead when numLoops > 1.

on the linux platform,evio works by epoll, and the epoll instances num = numLoops. Many epoll instances watch the same udp fd. This will cause thundering herd problems. so you found too much syscall.

tianlin commented 1 year ago

Hi: I found when set numLoops > 1, udp server with evio will be slower than numLoops = 1. Use profile, I found there is too much syscall in loopUDPRead(s, l, i, fd). I add a debug code fmt.Println("-- loop udp read --", l.idx) before loopUDPRead, then send just one udp msg to it, there are numLoops's print in stdout. I change the LoadBalance to AtLeastConnections / RoundRobin, there is also more call to loopUDPRead when numLoops > 1.

on the linux platform,evio works by epoll, and the epoll instances num = numLoops. Many epoll instances watch the same udp fd. This will cause thundering herd problems. so you found too much syscall.

thx