pombreda / gevent

Automatically exported from code.google.com/p/gevent
0 stars 0 forks source link

gevent monkey patching, involving Queue.Queue behaves improperly when run from nosetests #122

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
consider the following script:

import gevent.monkey
gevent.monkey.patch_all()
from gevent_zeromq import zmq
from gevent.queue import Queue
import threading
from Queue import Queue
import uuid
import time
q = Queue()
def first():
    print 'first'
    while True:
        try:
            q.get(timeout=10.0)
            print 'got'
        except:
            continue

def second():
    print 'hello'
    time.sleep(1)
    print 'hello'
    time.sleep(1)
    print 'hello'
    time.sleep(1)
    print 'hello'
    time.sleep(1)
    print 'hello'
    time.sleep(1)

def run_test():
    t2 = threading.Thread(target=second)
    t2.start()
    t = threading.Thread(target=first)
    t.start()
    t2.join()

if __name__ == "__main__":
    run_test()

if you  run that script with nosetests -s, it will lock up, if you run it with 
a regular python interpreter, it will not lock up.  if you remove from Queue 
import Queue, so that gevent.queue.Queue is used instead, it will not lock up.  
if you remove the timeout on the Queue.get() function, it will not lock up 
(Even though it blocks forever)

What is the expected output? What do you see instead?
This code sample should not lock up in any case.

What version of the gevent are you using?
0.13.6
On what operating system?
ubuntu 11.10
On what Python?
python 2.7
Please provide any additional information below.

I just did an additional test  - I think the issue is importing nose conflicts 
with some of gevents monkey patching, if you import nose before executing the 
script, it locks up.

Original issue reported on code.google.com by h...@continuum.io on 11 Apr 2012 at 6:23

GoogleCodeExporter commented 9 years ago
Can you try 1.0b2? The 0.13.6 has a known issues with monkey patching 
threading. 

You can get it from http://code.google.com/p/gevent/downloads/list

Original comment by Denis.Bi...@gmail.com on 11 Apr 2012 at 7:49

GoogleCodeExporter commented 9 years ago
sure - I don't perceive this as a gevent bug - if something imports something 
before you can run monkey patch, there's not much you can do right?

I just wanted to put it up so that at least if someone else runs into it, they 
can see this.  I'll run a test and let you know whatI  find.

Original comment by h...@continuum.io on 11 Apr 2012 at 7:51

GoogleCodeExporter commented 9 years ago
Ideally monkey patching should work even after the import (if possible). So if 
we can fix it we should.

Original comment by Denis.Bi...@gmail.com on 11 Apr 2012 at 7:53

GoogleCodeExporter commented 9 years ago
I'm not 100% sure this is the same, but this sounds very close to an issue I 
had with pycassa and gevent. The answer in that case was a secondary 
monkey-patching of the stdlib Queue to use the Queue from gevent as shown here:
https://github.com/phus/geventmonkey/blob/master/geventcassa.py

The geventmonkey lib has several other sets of monkey patches to help other 3rd 
party libraries work properly. Do those belong in gevent itself? Seems kind of 
a pain to have these separate, at the very least they should be in a blessed 
gevent-patch repo or something so that everyone ideally uses the same monkey 
patch.

Original comment by b...@groovie.org on 16 Apr 2012 at 8:11

GoogleCodeExporter commented 9 years ago
[deleted comment]
GoogleCodeExporter commented 9 years ago
[deleted comment]
GoogleCodeExporter commented 9 years ago
Migrated to http://github.com/SiteSupport/gevent/issues/122

Original comment by Denis.Bi...@gmail.com on 14 Sep 2012 at 10:52