search

jrkerns / pylinac

An image analysis library for medical physics
MIT License
146 stars 94 forks source link

Parallel test runner suggestion #349

Closed crcrewso closed 7 months ago

crcrewso commented 3 years ago

Is your feature request related to a problem? Please describe. There are a lot of unit tests in this repository, it takes a long long time to run them

Describe the solution you'd like Adding support for a unittest compatible parallel runner, like pytest-xdist would save time, without impacting existing expected workflows.

Describe alternatives you've considered We could ignore it, or consider a refactoring all tests to a different test runner. Neither is ideal.

Additional context I've tested a pytest.ini file that would allow users to choose pytest or ignore it and unittest will work as expected. This worked for me, both on the command line and with VS Code

adding pytest-xdist to the requirements.txt file is an optional extension of this proposal

[pytest]
addopts = -n auto  -ra -q
testpaths =
        tests_bank
        tests_basic
        tests_complete
python_files =
        test_*.py
        _test_*.py
jrkerns commented 3 years ago

Sounds good to me.

crcrewso commented 3 years ago

Right now, with python 3.8, on a 16 core system with 128 gb ram

pytest.ini:

[pytest]
addopts = -n auto
testpaths =
        tests_basic
        tests_complete
python_files =
        test_*.py
Command Results summary
pytest 74 failed, 594 passed, 3898 warnings, 149 errors in 39.86s
python -m unittest Ran 594 tests in 246.975s, FAILED (failures=3, errors=20)
python setup.py test Ran 594 tests in 252.176s, FAILED (failures=3, errors=10)

pytest appears to be picking up a few more tests, but I think it's a good start

jrkerns commented 3 years ago

Make a PR! This is good since we can still use unittest until this old dog (me) learns pytest.

crcrewso commented 3 years ago

Make a PR! This is good since we can still use unittest until this old dog (me) learns pytest.

Right now I cannot see any way to avoid needing to run python -m unittest first. Also there are some compatibility issues with 3.9 that I need to get documented and sent in. The next thing after finishing the results_data() work for the CT module

jrkerns commented 7 months ago

Set up a while ago. Works in all our CI/CD pipelines 👍https://github.com/jrkerns/pylinac/blob/5b040007f1150feb90d7affda3286aa1b2021c81/pyproject.toml#L56