eugeniy / pytest-tornado

A py.test plugin providing fixtures and markers to simplify testing of asynchronous tornado applications.
Apache License 2.0
121 stars 34 forks source link

IOLoop._instance is never set, causing hanging callbacks on multi-threaded tests #27

Open wouterdb opened 6 years ago

wouterdb commented 6 years ago

I noticed a subtle problem when using IOLoop.current(instance=True) in multi-threaded applications when using pytest-tornado:

If tornado starts the first IOLoop (on the main thread) it sets IOLoop._instance to point to this IOLoop However, the fixture never sets this field.

When calling IOLoop.current(instance=True) on any thread except for the thread in which the pytest-tornado ioloop fixture is executed, it will not find a current IOLoop for that thread and fall back to the global IOLoop._instance . If this field is absent, IOLoop assumes this is the first attempt to get an IOLoop and create a new IOLoop and set it as IOLoop._instance.

As such, all calls to IOLoop.current(instance=True) from other threads will get this IOLoop instance, which has not been started. All calls scheduled on that IOLoop will hang, causing tests to timeout.

Setting IOLoop._instance before the test and removing it afterwards resolves the issue.

(I hope my description is clear).

yen3 commented 6 years ago

I meet one problem. When I try to call tornado.process.Subprocess.wait_for_exit() function, the pytest-tornado will hang. I don't the problem is like what @wouterdb mentioned or not.

The test code is as the following:

from tornado import gen
from tornado.process import Subprocess

import pytest

@gen.coroutine
def echo_hello_world():
    return_code = yield Subprocess(args=["echo", "hello", "world"]).wait_for_exit()

    return return_code

@pytest.mark.gen_test
def test_cmd_1():
    yield echo_hello_world()

@pytest.mark.gen_test
def test_cmd_2():
    yield echo_hello_world()

The test will hang in test_cmd_2