pytest-dev / pytest-timeout

MIT License
206 stars 63 forks source link

`timeout_func_only = true` config does not work #142

Closed simon-liebehenschel closed 11 months ago

simon-liebehenschel commented 1 year ago

Reproducible code sample:

import asyncio
import pytest

@pytest.fixture()
async def my_fixture():
    await asyncio.sleep(5)

@pytest.mark.usefixtures("my_fixture")
@pytest.mark.timeout(2)
async def test_foobar():
   ...

pytest.ini

[pytest]
asyncio_mode = auto
timeout_func_only = true

Execution result

>           fd_event_list = self._selector.poll(timeout, max_ev)
E           Failed: Timeout >2.0s

Expected result

No errors.

Versions

simon-liebehenschel commented 1 year ago

I found that using @pytest.mark.timeout(2, func_only=True) (with the func_only=True) will not kill a fixture. But now I wonder then why a (useless?) timeout_func_only = true option exists for "pytest.ini".

RomanLeca commented 1 year ago

I can't get this working with pytest.ini or the marker over a single test. It seems I get stuck after it times out, but the teardown never happens.

flub commented 11 months ago

I think this works as intended. func_only only applies the timeout to the fuction and not to any fixtures.

simon-liebehenschel commented 11 months ago

I think this works as intended. func_only only applies the timeout to the fuction and not to any fixtures.

@flub Please take a look at my example above more carefully.

There is a section in the docs that says: Avoiding timeouts in Fixtures. But actually there is no way to avoid a timeout in a test if the test uses a slow fixture.

To clarify once more:

Please correct me if I am wrong or adapt my example above. Maybe we can include it then to the documentation as an example.

flub commented 11 months ago

ok, seems like i misread the issue.

Are you sure this isn't an asyncio issue though? Can you try this out without asyncio? I've never used asyncio and there's another issue about asyncio iirc

simon-liebehenschel commented 11 months ago

Thank you for the reply.

I debugged this by myself a little and I found that applying @pytest.mark.timeout on a test overrides timeout_func_only = true defined in the pytest.ini (for the marked test).

I expected that timeout_func_only = true defined in the pytest.ini will keep working even if I have @pytest.mark.timeout applied for my test.

The solution is that any user must define @pytest.mark.timeout with the func_only argument to keep the behaviour for a marked function. Example:

import time, pytest

@pytest.fixture
def slow():
    time.sleep(3)

@pytest.mark.timeout(  # We marked a test so for this test `pytest.ini` config will not work anymore.
    1,
    func_only=True  # This is mandatory if you have `timeout_func_only = true` and you  want to keep it for this test.
)
def test_foo(slow):
    pass

Maybe this should be mentioned in the docs, but there is nothing to fix. I am closing this.

flub commented 11 months ago

On Mon 09 Oct 2023 at 12:03 -0700, AIGeneratedUsername wrote:

Thank you for the reply.

I debugged this by myself a little and I found that applying @.***to a test overridestimeout_func_only = truedefined in thepytest.ini`.

Oh, good catch!

I expected that timeout_func_only = true defined in the pytest.ini will keep working even if I have @.***` applied for my test.

The solution is that any user must define @.***with thefunc_only` argument to keep the behaviour for a marked function.

I don't think this is entirely unreasonable to expect. Perhaps you could look into how to make it preserve that setting from the ini file in a more reasonable way and make a PR?

Maybe this should be mentioned in the docs, but there is nothing to fix. I am closing this.

Or at the very least please do a PR adding this to the docs in the README please.

Thanks!