It seems as though the thread module is not patched on eventlet workers, which causes socket errors w/ pymysql + django. I see in the gunicorn source that everything is patched except "os", so the thread module should be patched in all workers correct?
However, the thread module seems not to be patched when I use a pymysql backend w/ django. Manually patching the thread module in settings.py fixes socket errors, however after 30 minutes of continuous load, nginx logs a mix of 499s an 502s with no errors in the gunicorn.log while nginx logs show "upstream timed out (110: Connection timed out) while reading response header from upstream".
I'm wondering if I should even be explicitly patching the thread module in the application code, since it's already done by gunicorn.
example socket error when thread module is not explicitly patched:
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/query.py", line 364, in get
num = len(clone)
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/query.py", line 88, in len
self._result_cache = list(self.iterator())
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/query.py", line 294, in iterator
for row in compiler.results_iter():
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/sql/compiler.py", line 763, in results_iter
for rows in self.execute_sql(MULTI):
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/sql/compiler.py", line 818, in execute_sql
cursor.execute(sql, params)
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/backends/util.py", line 40, in execute
return self.cursor.execute(sql, params)
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/redstone/backends/pymysql2/base.py", line 135, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.7/dist-packages/PyMySQL-0.5-py2.7.egg/pymysql/cursors.py", line 121, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.7/dist-packages/PyMySQL-0.5-py2.7.egg/pymysql/connections.py", line 189, in defaulterrorhandler
raise Error(errorclass, errorvalue)
Error: (<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'read'",))
It seems as though the thread module is not patched on eventlet workers, which causes socket errors w/ pymysql + django. I see in the gunicorn source that everything is patched except "os", so the thread module should be patched in all workers correct?
However, the thread module seems not to be patched when I use a pymysql backend w/ django. Manually patching the thread module in settings.py fixes socket errors, however after 30 minutes of continuous load, nginx logs a mix of 499s an 502s with no errors in the gunicorn.log while nginx logs show "upstream timed out (110: Connection timed out) while reading response header from upstream".
I'm wondering if I should even be explicitly patching the thread module in the application code, since it's already done by gunicorn.
Thank you so much for your time!
gunicorn version: 0.17.2 django: 1.4.0 os: ubuntu 12.04
example socket error when thread module is not explicitly patched:
File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/query.py", line 364, in get num = len(clone) File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/query.py", line 88, in len self._result_cache = list(self.iterator()) File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/query.py", line 294, in iterator for row in compiler.results_iter(): File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/sql/compiler.py", line 763, in results_iter for rows in self.execute_sql(MULTI): File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/models/sql/compiler.py", line 818, in execute_sql cursor.execute(sql, params) File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/libs/django/db/backends/util.py", line 40, in execute return self.cursor.execute(sql, params) File "/mnt/ig/ed/persist/ed-persist-trunk-s23930-j8596/redstone/backends/pymysql2/base.py", line 135, in execute return self.cursor.execute(query, args) File "/usr/local/lib/python2.7/dist-packages/PyMySQL-0.5-py2.7.egg/pymysql/cursors.py", line 121, in execute self.errorhandler(self, exc, value) File "/usr/local/lib/python2.7/dist-packages/PyMySQL-0.5-py2.7.egg/pymysql/connections.py", line 189, in defaulterrorhandler raise Error(errorclass, errorvalue) Error: (<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'read'",))