kevlened / pytest-parallel

A pytest plugin for parallel and concurrent testing
https://github.com/browsertron/pytest-parallel/issues/104#issuecomment-1293941066
MIT License
313 stars 59 forks source link

Fixture Scope: All scopes behave as if function scoped, even with 1 process and 1 thread #85

Open bobobox opened 3 years ago

bobobox commented 3 years ago

Versions:

Python 3.8.0, 3.7.5 pytest==6.0.1 pytest-parallel==0.1.0

Description

To clarify, I understand that session is not shared between processes or threads (which seems to be a common point of confusion when using either pytest-parallel or pytest-xdist), but what I'm seeing is that, even with --workers 1 --tests-per-worker 1 (and everything beyond), fixtures scoped to session, class, etc all get re-run for every single test.

For the simplest example:

conftest.py

import pytest
import uuid

@pytest.fixture("class")
def session_scoped_fixture():
    return uuid.uuid4()

test.py

import pytest

def test1(session_scoped_fixture):
    assert 1 == session_scoped_fixture

def test2(session_scoped_fixture):
    assert 1 == session_scoped_fixture

def test3(session_scoped_fixture):
    assert 1 == session_scoped_fixture

def test4(session_scoped_fixture):
    assert 1 == session_scoped_fixture

Expected results

Since the fixture returns a unique UUID when run, one would expect output like this:

$ pytest --workers 2 test.py # OR pytest --workers 1 --tests-per-worker 2
...
==================================== short test summary info ====================================
FAILED test.py::test1 - AssertionError: assert 1 == UUID('deadbeef-40f1-4042-a047-3f2bf7fc6ac1')
FAILED test.py::test2 - AssertionError: assert 1 == UUID('112294dc-7949-475c-9b32-874e1fa99164')
FAILED test.py::test3 - AssertionError: assert 1 == UUID('deadbeef-40f1-4042-a047-3f2bf7fc6ac1')
FAILED test.py::test4 - AssertionError: assert 1 == UUID('112294dc-7949-475c-9b32-874e1fa99164')

ie. there are two unique IDs, for two sessions, one per worker (or thread).

Actual results

However, whenever pytest-parallel is involved, even if set to --workers 1 --tests-per-worker 1 (which should mean no concurrency or parallelism at all), every single test function appears to effectively have its own session:

$ pytest --workers 2 test.py # OR pytest --workers 1 --tests-per-worker 2
...
==================================== short test summary info ====================================
FAILED test.py::test1 - AssertionError: assert 1 == UUID('deadbeef-40f1-4042-a047-3f2bf7fc6ac1')
FAILED test.py::test2 - AssertionError: assert 1 == UUID('112294dc-7949-475c-9b32-874e1fa99164')
FAILED test.py::test3 - AssertionError: assert 1 == UUID('2fac72e8-ab02-4f41-8403-232119f70bad')
FAILED test.py::test4 - AssertionError: assert 1 == UUID('35d7f4ac-4804-40a6-9880-6153db735de3')

ie. There are four unique UUIDs, not the expected two.

For what it's worth, I don't have this issue with pytest-xdist: When running with -n 2 for example, there are two unique UUIDs among the four tests.

Is this expected? It seems like a bug, and I can't find it documented, but perhaps I'm misunderstanding something about how this all works.

vmred commented 3 years ago

Any updates on this?