golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
122.89k stars 17.52k forks source link

runtime: netpollWaiters typically not decremented #33624

Open rutsky opened 5 years ago

rutsky commented 5 years ago

What is the semantic of runtime.netpollWaiters? If it should track number of goroutines waiting for poll result, then it is implemented incorrectly (at least on Linux with epoll).

I see that runtime.netpollWaiters is incremented every time new goroutine getting blocked on polling:

https://github.com/golang/go/blob/f686a2890b34996455c7d7aba9a0efba74b613f5/src/runtime/netpoll.go#L354-L363

and decremented only in func netpollgoready(gp *g, traceskip int):

https://github.com/golang/go/blob/f686a2890b34996455c7d7aba9a0efba74b613f5/src/runtime/netpoll.go#L365-L368

Looks like netpollgoready() is called only from internal/poll.runtime_pollSetDeadline(), i.e. in some codepaths related to setting polling deadlines:

https://github.com/golang/go/blob/f686a2890b34996455c7d7aba9a0efba74b613f5/src/runtime/netpoll.go#L204-L205

And most frequently parked goroutine waiting for poll result is being awakened somewhere in runtime.findrunnable():

https://github.com/golang/go/blob/61bb56ad63992a3199acc55b2537c8355ef887b6/src/runtime/proc.go#L2210-L2221

or runtime.pollWork():

https://github.com/golang/go/blob/61bb56ad63992a3199acc55b2537c8355ef887b6/src/runtime/proc.go#L2395-L2409

Apparently atomic.Load(&netpollWaiters) > 0 condition in the referenced above runtime.findrunnable() and runtime.pollWork() functions is always true, as soon as as single goroutine will wait for poll result and get awakened from those functions.

I verified that runtime.netpollWaiters is increased with each wait of a goroutine on network in an example of handling TCP conection:

// tcp-server.go

package main

import (
        "bufio"
        "fmt"
        "log"
        "net"
        "strings"
)

func main() {

        fmt.Println("Launching server...")

        ln, _ := net.Listen("tcp", ":8081")

        conn, _ := ln.Accept()

        for {
                message, err := bufio.NewReader(conn).ReadString('\n')
                if err != nil {
                        log.Fatal(err)
                }
                fmt.Print("Message Received:", string(message))
                newmessage := strings.ToUpper(message)
                conn.Write([]byte(newmessage + "\n"))
        }
}
$ dlv debug tcp-socket.go
Type 'help' for list of commands.
(dlv) p runtime.netpollWaiters
0
(dlv) c
Launching server...
received SIGINT, stopping process (will not forward signal)
> runtime.epollwait() /usr/lib/golang/src/runtime/sys_linux_amd64.s:675 (PC: 0x4619e0)                                                                               
Warning: debugging optimized function
(dlv) p runtime.netpollWaiters
1
(dlv) c
Message Received:message 1 from netcat
Message Received:message 2 from netcat
received SIGINT, stopping process (will not forward signal)
> runtime.epollwait() /usr/lib/golang/src/runtime/sys_linux_amd64.s:675 (PC: 0x4619e0)
Warning: debugging optimized function
(dlv) p runtime.netpollWaiters
4
(dlv) 
$ go version
go version go1.12.5 linux/amd64

Snippet from go env:

$ go env
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
andybons commented 5 years ago

@aclements