look into TCP fastopen and TCP_QUICKACK

andrewrk commented 1 year ago

Extracted from #13980.

Here's an example of using TCP fastopen to send data along with the tcp connection:

    struct msghdr mh = {
        .msg_name = (void *)sa,
        .msg_namelen = sl,
        .msg_iovlen = 2,
        .msg_iov = (struct iovec [2]){
            { .iov_base = (uint8_t[]){ ql>>8, ql }, .iov_len = 2 },
            { .iov_base = (void *)q, .iov_len = ql } }
    };
    int fd = socket(family, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0);
    if (!setsockopt(fd, IPPROTO_TCP, TCP_FASTOPEN_CONNECT, &(int){1}, sizeof(int))) {
        r = sendmsg(fd, &mh, MSG_FASTOPEN|MSG_NOSIGNAL);

TCP_QUICKACK is another option that could be enabled in setsockopt. Both options are intended to reduce latency.

This issue is to change the std lib API for making an HTTPS request to take advantage of these things, and see if we can reduce the latency of establishing a connection. For HTTP GET requests it should do the same but with the HTTP header instead of the TLS ClientHello.

patryk4815 commented 1 month ago

TCP_FASTOPEN feature is no longer used by any major browsers due to privacy concerns. However, for internal communication within DC environments, these privacy issues may be less relevant. Probably no one is waiting for this feature :)

rofrol commented 1 month ago

More controversially, I suspect that Nagle’s algorithm just isn’t needed on modern systems, given the traffic and application mix, and the capabilities of the hardware we have today. In other words, TCP_NODELAY should be the default. That’s going to make some “write every byte” code slower than it would otherwise be, but those applications should be fixed anyway if we care about efficiency.

As this has gone around the internet, a number of folks have asked about TCP_QUICKACK. I don’t tend to reach for it for a few reasons, including lack of portability, and weird semantics (seriously, read the man page). The bigger problem is that TCP_QUICKACK doesn’t fix the fundamental problem of the kernel hanging on to data longer than my program wants it to. When I say write(), I mean write().

ziglang / zig

look into TCP fastopen and TCP_QUICKACK #14173