haskell-distributed / network-transport-tcp

TCP Realisation of Network.Transport
http://haskell-distributed.github.com
BSD 3-Clause "New" or "Revised" License
30 stars 27 forks source link

TCP_NODELAY=True would be a better default #66

Closed nh2 closed 1 month ago

nh2 commented 7 years ago

network-transport-tcp has tcpNoDelay = False by default.

Enabling Nagle's algorithm (not setting TCP_NODELAY) is not appropriate for cases where many multithreaded users multiplex over a single TCP connection, as is done in network-transport-tcp.

As described on e.g. https://eklitzke.org/the-caveats-of-tcp-nodelay:

Multi-threaded programs that share sockets between threads without locks are particularly susceptible to problems related to Nagle’s algorithm. This is because even if the application is structured so that a single thread doesn’t do the write-write-read pattern, if there is one writer thread and another reader thread then the pattern can still manifest. This is one good argument for not sharing sockets between threads without mutexes; usually a better option is to use an event loop.

By now I have seen many performance issues caused by this in real-world uses of network-transport-tcp, e.g.

In my opinion, network-transport-tcp by design needs TCP_NODELAY because it is multiplexing, and thus the actual senders and receivers have no control over in what interleaving messages are sent and received.

Further, it already uses sendMany where possible to combine as much data as possible into a single send, so the additional kernel-space batching performed by Nagle's algorithm does not add much on top (apart from the dreaded 40 ms send delay).

nh2 commented 7 years ago

Python 3.6 made TCP_NODELAY the default for asyncio: https://bugs.python.org/issue27456

LaurentRDC commented 1 month ago

Thank you for your patience! The change has been made and is now part of network-transport-tcp 0.8.4, released today.

nh2 commented 1 month ago

@LaurentRDC Thanks!