x/crypto/ssh: Close hangs with remote forward if TCP session is hung

[spikecurtis]

Go version

x/crypto v0.27.0

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/spike/Library/Caches/go-build'
GOENV='/Users/spike/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/spike/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/spike/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/spike/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/spike/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.6'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/spike/repos/coder/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/53/zffdtv3x7lg_pyhrk85p7_nw0000gn/T/go-build1889086903=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

Connect via SSH and set up remote forwarding of a TCP port or Unix socket (both exhibit the problem) via a call to Listen().

Break the networking: unplug Ethernet cable, disable WiFi, etc.

Close the Listener.

What did you see happen?

Closing the Listener hangs indefinitely, until the TCP socket times out, which could be hours or days.

What did you expect to see?

Closing the Listener times out with some reasonable timeframe, or accepts a timeout (e.g. via a `Context).

Typical use patterns would be:

client, err := ssh.Dial("tcp", server, cfg)
if err != nil {
    return err
}
defer client.Close()
l, err := client.Listen("tcp", addr)
if err != nil {
    return err
} 
defer l.Close()
waitForConnectionOrClose(l)

With the defers stacked like this, kind of the only sensible way, when you try to close the listener gets closed first, and you can hang indefinitely.

I've been able to work around the issue by building code that sets timeouts for the listener to close and if it fails, starts closing stuff from the bottom up, but it's much more difficult than being able to set a timeout for the listener to close down.

[gabyhelp]

Related Issues and Documentation

• ssh: hangs if remote side closes connection #5877 (closed)
• x/crypto/ssh: cannot close ssh session #52290
• crypto/tls: Conn.Close doesn't unblock stuck conn.Read #23518 (closed)
• x/crypto/ssh: Dial function with ClientConfig Timeout got stuck #46536 (closed)
• x/crypto/ssh: Dial hangs in kexLoop indefinitely - ignoring ClientConfig.Timeout #51926
• x/crypto/ssh: listening on remote address blocks indefinitely when address doesn't contain a host part #33227
• x/crypto/ssh: ssh.Dial can hang indefinitely #15113 (closed)
• x/crypto/ssh: calling http.Hijack over forwarded SSH connection blocks forever #68621 (closed)
• x/crypto/ssh: Session.Wait() hangs while reading from msg channel after sending EOF to os.Stdin #6088 (closed)
• x/crypto/ssh: cannot send a Signal to remote process without Pty #48715

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[cagedmantis]

cc @drakkan @golang/security