reading the response is taking too long - 10 minutes

[itamarzil123]

I am using native_tls::{TlsConnector, TlsStream} as part of building a browser project in Rust. reader.read_to_end(...) - this read operation hangs for about 10 minutes - and only after this enormous delay, the html contents of the URL (www.bbc.com for example) is resolved. for comparison - in browsers like chrome - takes 2 seconds for www.bbc.com, in curl takes 2-5 seconds. any thoughts on why this process my take so long ? (nothing serious on the wireshark logs, handshake times seems reasonable) simplified version of my code:

let mut response = Vec::new();
let mut tcp_stream = TcpStream::connect((ip, protocol_port)).expect("Error...");
let connector = TlsConnector::new().expect("Error creating TLS connector");
let mut tls_stream = connector.connect(host.as_str(), tcp_stream).expect("Error connecting with TLS");
tls_stream.write_all(request.as_bytes()).expect("Error....");
let mut reader = io::BufReader::new(tls_stream);
let mut buffer = [0; 4096]; 
reader.read_to_end(&mut response).expect("error...");
note: tried with a buffer, tried with 'read' instead of read_to_end

[sfackler]

read_to_end reads until the connection closes. You need to parse the response to determine when it is complete.

[itamarzil123]

@sfackler thank you for your answer. as I mentioned the code I shosed is a simplified part. the rest of the code includes: let response_str = String::from_utf8_lossy(&response); println!("response_str: {}", response_str); and also writing to files. the operation is successfull, however it takes 10 minutes to complete (unlike using curl / browser which takes 2 seconds) I tried buffering and didn't work. another thing I noticed - looking at the Wireshark logs: when I do: curl, I see no Encrypted Alert, the handshake and the application data all working great, but in my Rust program when I send the https request using native-tls I see that the read operation takes long and also I see: "Encrypted Alert" Content Type: Alert 21 in wireshark after those 10+- minutes and then the response is successful any thoughts ? or even how to debug ?

[itamarzil123]

I will also add that with Rustls library there were no issues (maybe related to the fact TLS 1.3 is used ?) anyway I prefer the API of native-tls so I hope I will resolve it

[sfackler]

Again, you cannot just read until the socket closes to get a complete HTTP response: https://en.wikipedia.org/wiki/HTTP_persistent_connection