search

benoitc / gunicorn

gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.
http://www.gunicorn.org
Other
9.84k stars 1.75k forks source link

High CPU usage with timeout 0 #567

Closed theflow closed 11 years ago

theflow commented 11 years ago

Hi,

I was running gunicorn with timeout set to 0 and noticed high CPU usage from all the workers, all the time. I can easily reproduce this by running the test.py form the documentation page like this:

gunicorn --workers=2 --timeout=0 test:app

on Mac and on Ubuntu 12.04 using Python 2.7.4

tilgovi commented 11 years ago

This is because timeout=0 does not mean infinite timeout. It means 0 timeout. Your workers are timing out constantly and being rebooted. Don't do this.

georgexsh commented 11 years ago

@tilgovi f6f118c has no help on this?

theflow commented 11 years ago

I assumed from https://github.com/benoitc/gunicorn/pull/370 that the intended behavior is to have no timeout in this case.

tilgovi commented 11 years ago

Hmm. I missed that change. I've reopened this so we can understand the cause.

benoitc commented 11 years ago

@theflow what is your gunicorn version?

georgexsh commented 11 years ago

@theflow it will be very helpful if you post the debug log and your system configuration (gunicorn/python version etc.)

theflow commented 11 years ago

Here is the debug log: https://gist.github.com/theflow/5965651

I can reproduce this on Ubuntu 12.04 using Python 2.7.3 and on Mac OS X 10.8.4 using Python 2.7.4. Gunicorn is version 17.5 in both cases.

benoitc commented 11 years ago

I can reproduce it using the test application :

The command line $ gunicorn -w3 --timeout=0 test:app produces such results:

   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                                                                                                    
23461 benoitc   20   0 54828   9m 1500 R  86.3  0.1   0:59.46 gunicorn: worke                                                                                                                            
23462 benoitc   20   0 54828   9m 1500 R  84.6  0.1   0:59.29 gunicorn: worke                                                                                                                            
23460 benoitc   20   0 54828   9m 1500 R  79.9  0.1   0:59.28 gunicorn: worke                                                                                                                            
23455 benoitc   20   0 54828  11m 3348 S   0.0  0.1   0:00.36 gunicorn: maste

note that the master is fine.

benoitc commented 11 years ago

@theflow can you test the change above and let me know?

theflow commented 11 years ago

yeah, that fixed it for me. Is there any side effect of putting the 0.5 timeout on the select call?

benoitc commented 11 years ago

quite none. The question is more why do you want a timeout of 0 ? :)

georgexsh commented 11 years ago

@benoitc since this self.timeout patch only affect select call, is it better to re-write like this

timeout = self.timeout or 0.5 # COMMENTS
ret = select.select(self.sockets, [], self.PIPE, timeout)
benoitc commented 11 years ago

@georgexsh can you submit a PR with this change?