monkey.patch_thread silently breaks threading.Timer behaviour so the Timer no longer fires.

[eplodn]

• gevent version: '1.2.2'
• Python version: Python 3.4.3 (default, Nov 28 2017, 16:41:13), also happens on 3.6
• Operating System:Ubuntu 14.04.5 LTS, also happens on 16.04

Description:

I am writing a simple command interpreter with a timer to kill the interpreter if not used for some time. However, because of the usage of gevent monkey patching, the timer no longer fires.

Attached is the minimal example demonstrating this.

If the gevent.monkey.patch_thread() line is commented out, the Timer will fire in 1 second, and the line 'Foo!' will be printed. If the gevent.monkey.patch_thread() line is enabled, the Timer will never fire.

What I've run:

import gevent.monkey
gevent.monkey.patch_thread() # <- comment out this line for the timer to start firing

import cmd, sys, threading

def foo():
    print("Foo!")

class MyShell(cmd.Cmd):
    prompt = '(myshell) '
    _timer = None
    def _start_idle_timer(self):
        if self._timer: self._timer.cancel()
        self._timer = threading.Timer(1, foo)
        self._timer.daemon = True
        self._timer.start()
    def do_something(self, arg):
        print('Doing Something')
    def postcmd(self, stop, line):
        self._start_idle_timer()
        return stop
    def cmdloop(self):
        self._start_idle_timer()
        cmd.Cmd.cmdloop(self)

MyShell().cmdloop()

[jamadden]

Thank you for your report.

This actually doesn't have anything to do with timers. What's going on is that Cmd (and thus MyShell) is making a blocking operating system call via input and preventing the gevent event loop from running at all. To put it another way, it's not cooperative and no other greenlets can run while it is blocking. You can see this in the traceback if you hit Ctrl-C while the program is sitting there:

  File "/tmp/test.py", line 33, in cmdloop
    cmd.Cmd.cmdloop(self)
  File "//3.6/lib/python3.6/cmd.py", line 126, in cmdloop
    line = input(self.prompt)

The solution to make it cooperative is two-fold: pass cooperative versions of stdin and stdout and ask it not to use the input function:

# Create cooperative versions for this object
from gevent.fileobject import FileObjectThread
stdin = FileObjectThread(sys.stdin)
stdout = FileObjectThread(sys.stdout)
shell = MyShell(stdin=stdin, stdout=stdout)
# Make it use those instead of `input`
shell.use_rawinput = 0
# Now this will cooperate with the event loop
shell.cmdloop()

(Using a custom stdin/out is documented here for Cmd, and the fact that sys.std[in|out] is not patched by patch_all on Python 3---hence the need to pass specific objects---is documented here.)

[eplodn]

Thanks a ton @jamadden!

Any idea how can I use gnureadline completer if I'm not using raw input but rather wrapping the stdin with FileObjectThread? gnureadline is not happy if sys.stdin != sys.__stdin__.

[jamadden]

Sorry, no idea.