[Advisory] IPytest + pytest-xdist does not work (`ExitCode.NO_TESTS_COLLECTED: 5`)

[keen85]

pytest-xdist is a plugin for pytest allowing to run tests in parallel.

I installed it via pip but I can't get it to run in combination with IPytest. I allways get ExitCode.NO_TESTS_COLLECTED: 5

I tried so far creating a ipynb notebook with the following cells:

Setup

import time
import ipytest

def test_first():
    time.sleep(3)
    assert True

def test_second():
    time.sleep(3)
    assert True

def test_third():
    time.sleep(3)
    assert True

Test 1: plain ipytest → works ✅

ipytest.autoconfig(addopts=[])
ipytest.run()

======================================= test session starts =======================================
platform win32 -- Python 3.10.10, pytest-7.2.2, pluggy-1.0.0
rootdir: ...
plugins: anyio-3.6.2, xdist-3.2.0
collected 3 items

t_bcf84451f8db4a55bde8bada6fc2f412.py 
.
.
.
                                                    [100%]

======================================== 3 passed in 9.05s ========================================
<ExitCode.OK: 0>

Test 2: ipytest + pytest-xdist → no tests are executed 🛑

ipytest.autoconfig(addopts=["--numprocesses=2"])
ipytest.run()

======================================= test session starts =======================================
platform win32 -- Python 3.10.10, pytest-7.2.2, pluggy-1.0.0
rootdir: ...
plugins: anyio-3.6.2, xdist-3.2.0
gw0 I / gw1 I
gw0 [0] / gw1 [0]

====================================== no tests ran in 0.71s ======================================
<ExitCode.NO_TESTS_COLLECTED: 5>

What am I doing wrong?

[chmp]

Hi @keen85

Thanks for the report. With only a cursory glance, it seems like xdist could be hard to support: the plugin seems to start separate processes for parallelization. However a lot of the ipytest functionality relies on modifying the current interpreter state. I fear that there could be no way to pass forward this modified state to the started processes.

I will have a closer look at how python-xdist works in the coming days and investigate in detail what's going on.

[chmp]

@keen85 unfortunately I'm pretty sure it's not possible to support pytest-xdist in ipytest. In their "how it works" section, it states explicitly that the collection is done in each worker thread. However, the tests defined inside the notebook are only available in the main process.

[keen85]

@chmp thanks for taking your time and digging into it