curl / curl

A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features
https://curl.se/
Other
35.77k stars 6.41k forks source link

WebSocket Failure Doesnt Clear socket. #13829

Closed STRATZ-Ken closed 4 months ago

STRATZ-Ken commented 5 months ago

I did this

for (int i = 0; i < 1000; i++) // Example loop running 10 iterations
{
    Console.WriteLine($"Iteration {i + 1}");
    IntPtr curl = LibCurl.curl_easy_init();
    if (curl == IntPtr.Zero)
    {
        Console.WriteLine("Failed to initialize libcurl");
        continue;
    }

    LibCurl.curl_easy_setopt(curl, LibCurl.CURLoption.CURLOPT_URL, "wss://123.com");
    LibCurl.curl_easy_setopt(curl, LibCurl.CURLoption.CURLOPT_CONNECT_ONLY, 2L);

    CURLcode result = LibCurl.curl_easy_setopt(curl, LibCurl.CURLoption.CURLOPT_SSL_VERIFYPEER, 0L);
    if (result != CURLcode.CURLE_OK)
    {
        throw new Exception($"Failed to disable SSL peer verification with code: {result}");
    }

    result = LibCurl.curl_easy_setopt(curl, LibCurl.CURLoption.CURLOPT_SSL_VERIFYHOST, 0L);

    result = LibCurl.curl_easy_setopt(curl, LibCurl.CURLoption.CURLOPT_VERBOSE, 1L);
    if (result != CURLcode.CURLE_OK)
    {
        throw new Exception($"Failed to disable SSL host verification with code: {result}");
    }

    CURLcode res = LibCurl.curl_easy_perform(curl);
    if (res != 0)
    {
        Console.WriteLine("curl_easy_perform() failed: " + res);
        CloseSocket(curl);
        LibCurl.curl_easy_cleanup(curl);
        continue;
    }

    // WebSocket communication
    WebSocketCommunication(curl);

    // Cleanup
    LibCurl.curl_easy_cleanup(curl);

    // Optionally add a delay between iterations
    System.Threading.Thread.Sleep(2000); // Sleep for 1 second
}

I expected the following

During the above code test, I noticed that my application was crashing after just a short period.

When I attempt to go an unknown host such as wss://121231231312313.com the application will work as expected. The socket will close upon it failing to connect. However, when going to a domain where the connect could be alive but gets a 403 or 200 (but fails to connect, such as wss://123.com) the output will say failure but the socket will remain open on the host. After a while I get socket exhaustion.

Sorry, the code is in C#. If we wait about 180 seconds, they will eventually timeout but this is already too late when using a lot of calls.

image

Iteration 9
* Hostname 123.com was found in DNS cache
*   Trying 104.21.15.205:443...
* Connected to 123.com (104.21.15.205) port 443
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / [blank] / UNDEF
* Server certificate:
*  subject: CN=123.com
*  start date: May 23 21:54:25 2024 GMT
*  expire date: Aug 21 21:54:24 2024 GMT
*  issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1P5
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> GET / HTTP/1.1
Host: 123.com
Accept: */*
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: iezzJtLNANM8pYhmriJDTQ==

* old SSL session ID is stale, removing
< HTTP/1.1 200 OK
< Date: Wed, 29 May 2024 22:46:25 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< x-powered-by: PHP/8.1.27
< vary: Accept-Encoding
< platform: hostinger
< x-turbo-charged-by: LiteSpeed
< CF-Cache-Status: DYNAMIC
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=iaQng270GjkBmHcOF%2BnRAPo6REyoe%2BJ2eUgdnjCjyoc4FcS2zGNH2SEO12UlsdP8XTIqETEfaBHmJNhE1l0qjStjAgUhep0r2k6q%2FnwRMG87jaa6wneP%2Byda"}],"group":"cf-nel","max_age":604800}
< NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< Server: cloudflare
< CF-RAY: 88ba05385e0d4223-EWR
< alt-svc: h3=":443"; ma=86400
* Refused WebSockets upgrade: 200
* Closing connection

curl/libcurl version

Curl 8.5.0

operating system

Windows 11 OS.

dfandrich commented 5 months ago

I suspect this is by design: the connection is kept open to save time and resources in case the program wants to do further HTTP requests to the same host. You should be able to use CURLOPT_FORBID_REUSE to stop this from happening.

STRATZ-Ken commented 5 months ago

Hello Dan,

Thanks for such a quick reply.

Yes, I have tried that. It still opens sockets.

image

image

dfandrich commented 5 months ago

I don't know what that second image is showing. Of course it is still going to open a socket—it needs to talk to the server. The question is, does it close the socket afterward? Actually, in looking at the first log you shows, it even says "* Closing connection". How are you determining that the socket is actually kept open?

STRATZ-Ken commented 5 months ago

Hello Dan,

This is the windows resource monitor. It shows all open sockets for a specific application. When the socket closes, it will gray out. As you can see this is just looping and keeps creating more sockets forever. When I try to connect to wss://welcome.hello for example, it attempts to connect but instantly fails and the socket is immediately closed. (Hostname failure instead of http status code 200)

Sent from my iPhone

On May 29, 2024, at 7:27 PM, Dan Fandrich @.***> wrote:



I don't know what that second image is showing. Of course it is still going to open a socket—it needs to talk to the server. The question is, does it close the socket afterward? Actually, in looking at the first log you shows, it even says "* Closing connection". How are you determining that the socket is actually kept open?

— Reply to this email directly, view it on GitHubhttps://github.com/curl/curl/issues/13829#issuecomment-2138411834, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AF6S7HMJP6F4THWYM6QYNQTZEZP4ZAVCNFSM6AAAAABIP2F2QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZYGQYTCOBTGQ. You are receiving this because you authored the thread.Message ID: @.***>

dfandrich commented 5 months ago

Are you sure the socket isn't actually in FIN_WAIT1 or TIME-WAIT or some other closing-but-not-quite-closed-yet state?

bagder commented 5 months ago

As @dfandrich, I suspect this is the #13153 issue: the connections are not left alive, they are in some wait states.

STRATZ-Ken commented 5 months ago

Sorry, I am still debugging to hopefully collect more information.

STRATZ-Ken commented 4 months ago

So to the best of my knowledge, it is in-fact closing correctly. However the UI on Windows Resource Monitor is not function to the correct closing. It works fine with LibCurl POST/GET, but on Websockets it doesn't show it closing, however netstat -an does show it in TIME_WAIT status (Closed). I would consider this technically closed, though there is a very small bug I am not sure it warrants further investigation.