snapcore / spread

Spread - Convenient full-system test (task) distribution
GNU General Public License v3.0
99 stars 58 forks source link

client: fix hang on ssh.Dial() on reboot #160

Open mvo5 opened 1 year ago

mvo5 commented 1 year ago

This commit adds a thin wrapper around the real ssh.Dial() that additionally sets a deadline on the underlying connection.

It is needed because a ssh.Dial() can happens right after the reboot command is issued. The net.Dial() itself is successful but then then during the ssh session setup the TCP connection ends because of the reboot. The golang "ssh" package has no concpt of "ssh -o ServerAliveInterval=10" or simialr so the code will just hang in a read forever. This was observed running the spread "cerberus" tests on ubuntu 23.04.

Note that half of the function is just a copy of golang.org/x/crypto/ssh/client.go:func Dial() and only the conn.SetDeadline() bits are new.

See also e.g. https://github.com/golang/go/issues/51926 for various bugreports about the golang "ssh" package and hangs.