Implement RFC8305 Happy Eyeballs for dual stack IPv4 and IPv6 hosts

[nroi]

I've been looking at different Erlang HTTP client libraries, and haven't found a single one which supports happy eyeballs. Using IPv6 exclusively is still not an option since many hosts don't support it yet, and using IPv4 exclusively is also less than ideal: Many ISPs don't even give out new IPv4s to new customers and instead implement something like "Dual Stack Lite", which means that IPv6 works more reliably than IPv4 for those customers.

Some links:

• https://tools.ietf.org/html/rfc8305
• Happy Eyeballs in Go: https://github.com/golang/go/blob/master/src/net/dial.go#L46
• Curl: https://github.com/curl/curl/blob/001822153201e1e9b0246bc64a3797eb18f5ed61/lib/connect.h#L43
• Rust: https://github.com/hyperium/hyper/commit/02a9c29e2e816c8a583f65b372fcf7b8503e6bad

[essen]

This can be implemented on top of Gun, so I would suggest working on a reference implementation and then we can see about including it directly in Gun.

[essen]

Looked a bit more and it should be done in gun:open or in a new function wrapping it. First try IPv6 and then after a timeout try IPv4 and then picks up the first one that's up. Then select this one and return the info to the client so it can use the same on future attempts. Or maybe use an ets table for Gun to manage it on its own.

Not sure this implements the whole of RFC6555 but that's a start. Would love a proof of concept PR.

[nroi]

Keep in mind that RFC6555 has been obsoleted RFC8305. The algorithm is more complex than what you described, but I agree that an easier implementation of dual stack support is also good to have.

I've started to implement a work-in-progress sort-of prototype of RFC8305 in Elixir (https://github.com/nroi/eyepatch), it aims to be usable for many HTTP client libraries and I have tested it with hackney. The project is far from done, but I would like to share some insights:

• The full RFC is difficult to implement because it really aims to be able to provide the best possible outcome for all possible cases. For example, there are servers that have an AAAA-record set but when actually trying to connect via IPv6, the connection times out. With Happy Eyeballs, you will be able to connect to such servers via IPv4, and the whole process won't be significantly slower than with an IPv4-only client.
• The RFC requires making a new connection attempt while a previous connection attempt is still ongoing: This requires to keep a lot of state, e.g., which IPs have been tried and which attempts have failed.
• All of this is difficult to test: Because it involves DNS resolution, you would need integration tests (at least I wouldn't know how to test this functionality without an actual server with DNS entries).

I just glossed over the test directory in gun and it seems that gun currently doesn't have integration tests where connections and DNS resolutions are made to an external server, is that correct? Because that would be another argument for implementing just a simple version of dual stack support: adding many untested lines of code for a feature that is more of a nice-to-have than a necessity doesn't seem like a good idea.

[essen]

Gun just uses gen_tcp/inet which has those DNS related tests and I suspect a proper implementation of this RFC should sit there.

Or at least replicate a lot of the connect functionality, even more than I did in current Gun master which now separates domain lookup from connect and TLS handshake. Basically we could extend domain lookup to get all the records the RFC requires and then extend the connect to eventually try a second connection and do so asynchronously but that's quite a large amount of work so I was just thinking it'd be far easier to do it in front of Gun.

And because of how gen_tcp/inet is structured, it'd be far far easier to just try on inet6 first and after a timeout try inet (while leaving inet6 trying to connect) and just pick the first winner. Even if it doesn't implement the RFC fully. And it's unclear to me how much better a full implementation of the RFC would be compared to just doing this. Probably not much.

[essen]

By the way, related, gen_tcp:connect might itself benefit from sorting IP addresses, I've seen it with a [{127,0,0,1},{127,0,0,1}] list, implying it attempts to connect twice in some cases.

I'm also curious how the RFC behaves on Windows. In Erlang's case gen_tcp:connect will end up trying to connect 3 times to the given IP, and that is a Windows behavior. Don't know if that can be disabled without tweaking the registry.

[essen]

In case anyone really needing this comes upon this ticket, I have no plans to implement it at this time but would be willing to for a paid customer. Please see https://ninenines.eu/services/ for general information about that.

[dch]

I finally needed this, and it turns out @stolen has already implemented it years ago at https://github.com/yandex/inet64_tcp so I forked it, tidied it up for rebar3, and it works as promised. The magic bits are in here - https://github.com/skunkwerks/inet64_tcp/blob/main/src/inet64_tcp.erl#L72-L96

[essen]

I think happy eyeballs you must do queries concurrently, and then connects concurrently as well (after some wait time). First connect wins.

There's also https://github.com/hrefhref/gun/commit/c5d577e4b88b6a401ea15111d81240fb674b6717 as an attempt to deal with both 4 and 6.