Closed tailrecur closed 3 years ago
I am not able to naively reproduce using your exact sample code on my current machine:
* Trying 2606:2800:220:1:248:1893:25c8:1946...
* TCP_NODELAY set
* Connected to www.example.org (2606:2800:220:1:248:1893:25c8:1946) port 443 (#0)
I stepped through the code briefly and verified that libcurl was setting tcp_nodelay
internally in response to the Easy::tcp_nodelay
call. I can look into this deeper tonight.
@sagebind My gut feel is that I'm doing something wrong locally since this would be a bigger bug otherwise that would have been noticed earlier.
The only difference I can see in your example is that you are using IPv6
. Would it be possible to force Ipv4
usage?
I'll try to repeat this on a fresh server too.
I tried this on a fresh VM instance with the following spec:
root@latency-test:~/nodelay# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root@latency-test:~/nodelay# cat /etc/debian_version
10.8
root@latency-test:~/nodelay# uname -a
Linux latency-test 4.19.0-14-cloud-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux
I still get the behaviour described above. I'm trying to figure out how to force Debian to use IPv6 resolution.
@sagebind which OS flavour are you on?
My test was run on macOS 10.15.7. With libcurl 7.64.1.
@sagebind Thanks for the details!
I see the following behaviour on macOS 10.15.7 and libcurl 7.64.1.
Cargo.toml:
curl = "0.4.34"
TCP_DELAY is set correctly
Cargo.toml:
curl = { version = "0.4.34", features = ["static-curl"]}
TCP_DELAY is not set.
I see no difference in output from cargo tree
for both configurations. otool
confirms that one uses system libcurl
while the other does not.
Can you also try this with the static-curl
feature turned on please?
@tailrecur Great discovery! I ran your example with static-curl
enabled and was able to reproduce. I stepped through the program with a debugger and confirmed that CURLOPT_TCP_NODELAY
is still getting set, but curl was ignoring it. The reason why is revealed when you get to the definition of the internal tcpnodelay
function:
https://github.com/curl/curl/blob/2f33be817cbce6ad7a36f27dd7ada9219f13584c/lib/connect.c#L1055-L1074
static void tcpnodelay(struct Curl_easy *data, curl_socket_t sockfd)
{
#if defined(TCP_NODELAY)
curl_socklen_t onoff = (curl_socklen_t) 1;
int level = IPPROTO_TCP;
#if !defined(CURL_DISABLE_VERBOSE_STRINGS)
char buffer[STRERROR_LEN];
#else
(void) data;
#endif
if(setsockopt(sockfd, level, TCP_NODELAY, (void *)&onoff,
sizeof(onoff)) < 0)
infof(data, "Could not set TCP_NODELAY: %s\n",
Curl_strerror(SOCKERRNO, buffer, sizeof(buffer)));
#else
(void)data;
(void)sockfd;
#endif
}
However, the TCP_NODELAY
C macro is not defined for some reason in our current build process for static-curl
, likely some missing headers, so libcurl just silently does nothing. I will look into this.
It is also worth noting that looking for the TCP_NODELAY set
debug line is not sufficient to know whether TCP_NODELAY
is working or not because it was removed in curl 7.69.0:
https://github.com/curl/curl/commit/1b6cfb9d24cdcb822996bc23e2eb5a4cea73ec4e
@sagebind Thanks for the confirmation. Nice to know I wasn't dreaming this 😄
It is also worth noting that looking for the TCP_NODELAY set debug line is not sufficient to know whether TCP_NODELAY is working or not because it was removed in curl 7.69.0:
Nice to know. I was not too confident about relying on the debug logs in the original bug report and so wanted to independently confirm it with strace
and knetstat
just in case.
Thanks for the links to the curl code. I'll need to brush up on my C first to be able to tackle that.
Hello,
I'm building a testing-tool on top of
isahc
and couldn't figure out some odd behaviour around the TCP connection as detailed here and here.I finally narrowed it to down to the TCP socket not having the option
TCP_NODELAY
set on it even though the Rust program explicity sets this. I usedstrace
and knetstat to verify this. I'm able to reproduce the issue locally with the following simple example:Setup: OS: Debian 4.19.171-2 OpenSSL 1.1.1d
Cargo.toml:
Rust program:
Curl command(for comparison):
Note:
curl
hasTCP_NODELAY
on by default nowDifferences between
cargo run
andcurl
commandstdout
cargo run
curl
knetstat
cargo run
curl
strace
cargo run
curl
The
curl-rust
implementation here and here look the same as every other option and I've double-checked that the code121
is correct. I'm not sure why the options are not getting set on the socket though.Is my usage of
easy.tcp_nodelay(true)
incorrect in the sample program? Can you please try to reproduce this issue too?/cc @sagebind (sort of related to https://github.com/sagebind/isahc/issues/303)