pytest-dev / pytest-xdist

pytest plugin for distributed testing and loop-on-failures testing modes.
https://pytest-xdist.readthedocs.io
MIT License
1.45k stars 230 forks source link

pytest does not run tests in parallel and hangs after running first test #661

Open vishakha-vonage opened 3 years ago

vishakha-vonage commented 3 years ago

I'm facing a very weird issue. When I try to run my test in parallel- Only 1 browser opens even when I execute pytest -n 2 After first test is successful, pytest just hangs and it does not even timeout Has anyone faced this issue?

py.test -vv -n 2 =================================================================================================== test session starts =================================================================================================== platform darwin -- Python 3.7.3, pytest-5.4.1, py-1.10.0, pluggy-0.13.1 -- /// cachedir: .pytest_cache rootdir: //, inifile: pytest.ini plugins: xdist-1.31.0, allure-pytest-2.8.40, repeat-0.9.1, forked-1.1.3, timeout-1.4.2 [gw0] darwin Python 3.7.3 cwd: //* [gw1] darwin Python 3.7.3 cwd: /*/ collected 8 items [gw0] Python 3.7.3 (default, Apr 24 2020, 18:51:23) -- [Clang 11.0.3 (clang-1103.0.32.62)] [gw1] Python 3.7.3 (default, Apr 24 2020, 18:51:23) -- [Clang 11.0.3 (clang-1103.0.32.62)] gw0 [8] / gw1 [8] scheduling tests via MarkBasedScheduler

test1 [gw1] [ 12%] PASSED test1

It will get stuck after this.

eaa3 commented 2 years ago

I get the same issue sometimes, it happens randomly and it's quite hard to reproduce. Would it be related to multiprocessing issues using the fork sys call? https://bugs.python.org/issue40399 https://docs.python.org/3/library/multiprocessing.html#multiprocessing.set_start_method

My tests are multi-threaded and all sorts of weird hangs happen. Does pytests-xdist use fork system call when making tests parallel?

My setup is on linux (Ubuntu 20.04)

RonnyPfannschmidt commented 2 years ago

Pytest xdist starts a new non forked python, however execnet has a broken treading startup, so pytest control may not end up in the main thread in the worker

A fix is in very early planning, but may take a few months to delivery

gova-i267 commented 8 months ago

I have the same problem now, how is the problem progressing now?