Tests can not run in parallel, making "green" unable to speed up test running

[WolfgangFahl]

Describe the bug Testing with green fails

To reproduce

git clone https://github.com/linkml
cd linkml
pip install green
green

Expected behavior Tests should pass.

Screenshots Ran 476 tests in 135.054s using 16 processes

FAILED (errors=9, expected_failures=4, failures=18, passes=391, skips=54)

About your computer (if applicable, please complete the following information):

• OS: MacOs

[dalito]

Did you see the same errors with the standard unittest test runner that is used in CI? poetry run python -m unittest discover

[WolfgangFahl]

@dalito No - i 'll try now.

pip install poetry
poetry run python -m unittest discover

Creating virtualenv linkml-2wwaRrEC-py3.10 in /Users/wf/Library/Caches/pypoetry/virtualenvs
EE
======================================================================
ERROR: linkml (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: linkml
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/loader.py", line 470, in _find_test_path
    package = self._get_module_from_name(name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/wf/Documents/pyworkspace/linkml/linkml/__init__.py", line 4, in <module>
    import rdflib_shim
ModuleNotFoundError: No module named 'rdflib_shim'

======================================================================
ERROR: tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/loader.py", line 470, in _find_test_path
    package = self._get_module_from_name(name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/wf/Documents/pyworkspace/linkml/tests/__init__.py", line 6, in <module>
    from tests.utils.test_environment import MismatchAction
  File "/Users/wf/Documents/pyworkspace/linkml/tests/utils/test_environment.py", line 14, in <module>
    from linkml_runtime.linkml_model import linkml_files
ModuleNotFoundError: No module named 'linkml_runtime'

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=2)

[WolfgangFahl]

Then i tried:

export PYTHONPATH="."
python -m unittest discover 

the test are running and i can report the results in a few minutes ...

[WolfgangFahl]

Ran 462 tests in 1089.177s

FAILED (failures=1, errors=3, skipped=54, expected failures=4)

[WolfgangFahl]

Looks like there are dependencies between the tests that make the parallel approach of green fail which is a pitty since green is considerably faster (factor 8x) in this case.

[dalito]

Thanks for trying. I was speculating that green does something different than what is done in CI. I have also experienced failures linked to the test runner (in my case pytest and on Windows). When I repeatedly run parallel tests with pytest-xdist I get varying numbers of failures. Also when I run the test suite again (without cleaning the output), I get different errors than in clean state.