pytest timing is tricked by freezegun

[trimailov]

This is a copy of an issue in pytest, that needs coordination with freezegun.

--durations is tricked by freezegun, when freezegun is used in autouse fixture.

Example:

from freezegun import freeze_time

@pytest.yield_fixture(autouse=True)
def frozen_time():
    with freeze_time("2020-01-01T12:00:00Z"):
        yield

def test_my_django_view(admin_client):
    resp = admin_client.get(...)
    assert resp.status_code == 200

...

Command:

pytest --reuse-db -s -vv --durations=5

Result:

============================================= slowest 5 durations =============================================

1609479015.03s setup    test_my_django_view1
1609479014.85s setup    test_my_django_view2
1609479014.70s setup    test_my_django_view3
1609479014.60s setup    test_my_django_view4
1609479014.47s setup    test_my_django_view5

Versions (Python 3.10.2):

pytest                             7.1.2
pytest-django                      4.2.0
freezegun                          1.2.2

freezegun is latest version which has some work done regarding this: https://github.com/spulec/freezegun/pull/460

If every test function is decorated individually - then everything works as expected:

from freezegun import freeze_time

@freeze_time("2020-01-01T12:00:00Z")
def test_my_django_view(admin_client):
    resp = admin_client.get(...)
    assert resp.status_code == 200

...

Report:

3.75s setup    test_my_django_view1
0.49s call     test_my_django_view2
0.28s call     test_my_django_view3
0.27s call     test_my_django_view4
0.25s call     test_my_django_view5

Related issue: https://github.com/pytest-dev/pytest/issues/7764

Like in the linked issue - if you say that it's my problem of (mis)using freezegun (or that I need to speak with freezegun devs) - it's fine

[utapyngo]

https://github.com/spulec/freezegun/issues/433 is a related issue that is already solved but does not solve this one.

Here is a related comment: https://github.com/spulec/freezegun/issues/176#issuecomment-1149163454 that states the same problem and https://github.com/spulec/freezegun/issues/176#issuecomment-1149171726 that mentions pytest-durations.

[utapyngo]

Reading the source code of pytest-freezegun I figured out how it avoids tricking pytest timings: https://github.com/ktosiek/pytest-freezegun/blob/master/pytest_freezegun.py#L39-L41

So the solution is to call freeze_time with the ignore argument:

freeze_time("2020-01-01T12:00:00Z", ignore=['_pytest.runner', '_pytest.terminal'])

These should probably be added to the default ignore list.

According to the docs, the default ignore list includes ['_pytest.terminal.', '_pytest.runner.']. Probably, the dots at the end break it. I tried forking and adding them to DEFAULT_IGNORE_LIST but this breaks the tests: RecursionError: maximum recursion depth exceeded.