search

php-http / curl-client

cURL client
http://httplug.io
MIT License
443 stars 28 forks source link

process open fd table is full #38

Open traufeisen opened 6 years ago

traufeisen commented 6 years ago

During sendRequest the Client checks for the existence of a curl handle in the handle property. If there is one, it calls curl_reset on it and continues with that handle. The only way to destroy a curl handle is with the destructor of the Client.

The problem with this is how curl_resetworks. This function is only resetting the options, but not the underlying socket. From the curl manual

It does not change the following information kept in the handle: live connections, the Session ID cache, the DNS cache, the cookies and shares.

In long running scripts this leads to more and more open sockets in CLOSE_WAIT status. Depending on the remote servers. Until all allowed file descriptors are in use.

The workaround is to construct a new Client object for every request, so the destructor gets called when a request is finished. A better way would be to explicitely close the handle with curl_close after each request and use a new one for the next request.

mekras commented 6 years ago

OK, I'll check this in a few days.

bagaskarawg commented 4 years ago

Seems like this issue is what caused this problem to arise in our case: mmap cache can't open ... - Too many open files (pid 4690)

We are currently using this package which is provided by default by this package: https://github.com/geocoder-php/GeocoderLaravel. We are currently processing quite a bunch of request of around 10,000 requests each day and it stopped at around 400-500 requests, after searching for a solution online, I've found that it was due to the ulimit -n value which is currently set to 1024.

pdeans commented 3 years ago

In case this helps anyone else experiencing this issue, we solved this error in our application by setting the Connection: close header on the Client requests.