abourget / gevent-socketio

Official repository for gevent-socketio
http://readthedocs.org/docs/gevent-socketio/en/latest/
BSD 3-Clause "New" or "Revised" License
1.21k stars 331 forks source link

Leaks? #55

Closed jpellerin closed 12 years ago

jpellerin commented 12 years ago

It looks to me from running the simple chat example (and other applications) under Dozer (https://bitbucket.org/bbangert/dozer) that gevent-socketio is leaking a socketio.virtsocket.Socket and geventwebsocket.websocket.WebSocketHybi on every disconnect (along with a host of related/connected gevent/greenlet things).

The problem, as best as I can see, is that although kill() runs when the socket disconnects, the socket itself is never actually disposed of. From looking at the dozer output it seems like the problem may be that the environ dict is never deleted, so references remain to the websocket and socketio objects. I don't know if that's right (or even reasonable), or what to do to fix it.

jpellerin commented 12 years ago

The problem appears to be that the watcher greenlet never exits. Adding a 'break' on L396 of virtsocket.py stops the leaks, in my testing at least.

creotiv commented 12 years ago

After this patch i started to get this error on each request:

Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/gevent-0.13.6-py2.6-linux-x86_64.egg/gevent/pywsgi.py", line 438, in handle_one_response self.run_application() File "/usr/lib/python2.6/site-packages/gevent-0.13.6-py2.6-linux-x86_64.egg/gevent/pywsgi.py", line 424, in run_application self.result = self.application(self.environ, self.start_response) File "/data/www/ChatServer/scripts/main.py", line 177, in call '': ChatNamespace File "/usr/lib/python2.6/site-packages/gevent_socketio-0.3.5_rc1-py2.6.egg/socketio/init.py", line 60, in socketio_manage socket = environ['socketio'] KeyError: 'socketio'

: Failed to handle request: request = GET /socket.io/1/websocket/71276170094 HTTP/1.1 from ('125.165.165.207', 59576) application =
jpellerin commented 12 years ago

I didn't see that here, though I only tested with the example simple chat server and my app (https://github.com/leapfrogdevelopment/womack), which is also pretty simple. I'm also using a more recent gevent, if that makes a difference.

creotiv commented 12 years ago

i'm trying to implement chat with 9-12к load. And i can say that it realy leaking, for now i'm testing it with different configurations. Above error i'm getting from rime to time, but after your patch i starting to get it after each request.

jpellerin commented 12 years ago

I'm not sure why allowing the watcher greenlet for a disconnected request to exit would cause any issues for future requests. environ['socketio'] is set by the SocketIOHandler, which should be handling every request before the app code that runs socketio_manage gets a chance to do anything. I don't understand that at all, unless somehow the browser side is doing something crazy, or you're passing non-socketio urls to socketio_manage and it was seeming to work before somehow because of the old greenlets hanging around.

bbigras commented 12 years ago

I have a similar problem when trying the simple_pyramid_chat example with gevent-socketio==0.3.5-rc2.

D:\gevent-socketio\examples\simple_pyramid_chat>pserve development.ini
Starting server in PID 4276.
serving on http://0.0.0.0:6543
ERROR:waitress:Exception when serving /socket.io/1/
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\waitress-0.8.1-py2.7.egg\waitress\channel.py", line 329, in service
    task.service()
  File "C:\Python27\lib\site-packages\waitress-0.8.1-py2.7.egg\waitress\task.py", line 173, in service
    self.execute()
  File "C:\Python27\lib\site-packages\waitress-0.8.1-py2.7.egg\waitress\task.py", line 380, in execute
    app_iter = self.channel.server.application(env, start_response)
  File "C:\Python27\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\router.py", line 187, in __call__
    response = self.handle_request(request)
  File "C:\Python27\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\tweens.py", line 20, in excview_tween
    response = handler(request)
  File "C:\Python27\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\router.py", line 164, in handle_request
    response = view_callable(context, request)
  File "C:\Python27\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\config\views.py", line 333, in rendered_view
    result = view(context, request)
  File "C:\Python27\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\config\views.py", line 471, in _requestonly_view
    response = view(request)
  File "d:\gevent-socketio\examples\simple_pyramid_chat\chatter2\views.py", line 59, in socketio_service
    }, request=request
  File "build\bdist.win32\egg\socketio\__init__.py", line 60, in socketio_manage
    socket = environ['socketio']
KeyError: 'socketio'
jpellerin commented 12 years ago

The README for simple_pyramid_chat says that it should be started by running python serve.py, not pserve development.ini. The latter way it's not clear to me how (or if) the SocketIOServer ever gets involved, since that is only mentioned in serve.py. But I'm not too familiar with paste or pyramid so maybe I'm missing something.

In any case, when I run the app with python serve.py it appears to work correctly.

bbigras commented 12 years ago

You're right, sorry for the noise.