Too many open files

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Run a wsgi script to serve a tracking image (1x1 gif)
2. Hit server with full production load (~25 requests/second)
3. Drown in error logs

What is the expected output? What do you see instead?

I expect the small gif to be returned very quickly, but the response time 
jumps up to an unacceptable level.

Right at the start of the process:
$ time curl "http://.../tracker.gif"
GIF89a????????!??,D;
real    0m0.055s
user    0m0.003s
sys 0m0.005s

Again at T +1 minute
$ time curl "http://.../tracker.gif"
GIF89a????????!??,D;
real    0m6.366s
user    0m0.003s
sys 0m0.008s

What version of the product are you using? On what operating system?

Gevent 0.12.2
Ubuntu 9.10

Please provide any additional information below.

The service runs fine for like 30 seconds then it just starts spitting out: 

"[warn] accept_socket: bad accept: Too many open files"

Which definitely interferes with response time. Is there a way to tweak the 
gevent server or libevent to remedy this issue? I believe it occurs with long 
lasting http connections not being closed properly.

The server script simply uses the WSGIServer

from gevent.wsgi import WSGIServer
WSGIServer((host, port), application).serve_forever()

Original issue reported on code.google.com by justquick on 19 Mar 2010 at 2:46

[GoogleCodeExporter]

Just tried it again with the most recent tip and the test__http unittest passed 
fine but 
the problem still appears after like 30 seconds or so. I am using libevent-1.4-2

Original comment by justquick on 19 Mar 2010 at 3:00

[GoogleCodeExporter]

This does not seem like a gevent's issue to me. Instead it's a limitation set 
by your
operating system.

For example, take a look at these articles, where the author mentions how he 
raised
the relevant OS limits:
http://nichol.as/asynchronous-servers-in-python
http://nichol.as/benchmark-of-python-web-servers

Original comment by Denis.Bi...@gmail.com on 25 Mar 2010 at 8:19

• Changed state: Invalid

[GoogleCodeExporter]

Please re-open this issue.  This is not about raising operating system limits 
at all -- it s a bug in gevent's libevent usage.  The relevant HTTP sockets are 
NEVER closed by gevent after they have been used and the other side has closed 
the socket as well, therefore this causes a massive fd leak (which is easily 
hit in about 2 seconds).

Original comment by dragonf...@gmail.com on 15 Apr 2011 at 9:27

[GoogleCodeExporter]

Original comment by Denis.Bi...@gmail.com on 15 Apr 2011 at 9:41

• Changed state: New

[GoogleCodeExporter]

can you give an example script that reproduces it?

Original comment by Denis.Bi...@gmail.com on 15 Apr 2011 at 9:43

[GoogleCodeExporter]

thats not what went wrong on my end. the following OS tweaks fixed the problem 
for me. try em out

echo “10152 65535″ > /proc/sys/net/ipv4/ip_local_port_range
sysctl -w fs.file-max=128000
sysctl -w net.ipv4.tcp_keepalive_time=300
sysctl -w net.core.somaxconn=250000
sysctl -w net.ipv4.tcp_max_syn_backlog=2500
sysctl -w net.core.netdev_max_backlog=2500
ulimit -n 10240

commands taken from  http://nichol.as/benchmark-of-python-web-servers

Original comment by justquick on 18 Apr 2011 at 2:20

[GoogleCodeExporter]

OK, so it seems there's no issue with gevent here.

Original comment by Denis.Bi...@gmail.com on 20 Dec 2011 at 5:57

• Changed state: Invalid

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Migrated to http://github.com/SiteSupport/gevent/issues/18

Original comment by Denis.Bi...@gmail.com on 14 Sep 2012 at 10:51

• Changed state: Migrated