Fail gracefully on attempts to connect to an unspecified IP address

[Ralith]

Silent failure here has tripped up multiple users, most recently in https://github.com/quinn-rs/quinn/issues/1472.