NLeSC / python-template

Netherlands eScience Center Python Template
https://research-software-directory.org/software/nlesc-python-template
Apache License 2.0
163 stars 73 forks source link

Re-use baked project for all tests #353

Closed bouweandela closed 9 months ago

bouweandela commented 9 months ago

Description

Speed up the tests by re-using the baked project.

Also:

Output of pytest --durations=0 before (119 seconds):

============================================================================= slowest durations =============================================================================
10.56s setup    tests/test_project.py::test_bumpversion
10.56s setup    tests/test_project.py::test_tox
10.37s setup    tests/test_project.py::test_doctest_api_docs
10.36s setup    tests/test_project.py::test_coverage_api_docs
10.28s setup    tests/test_project.py::test_prospector
10.20s setup    tests/test_project.py::test_subpackage
10.19s setup    tests/test_project.py::test_isort_check
10.15s setup    tests/test_project.py::test_generate_api_docs
10.08s setup    tests/test_project.py::test_coverage
10.07s call     tests/test_project.py::test_tox
9.85s setup    tests/test_project.py::test_pytest
3.02s call     tests/test_project.py::test_prospector
0.72s call     tests/test_project.py::test_generate_api_docs
0.57s call     tests/test_project.py::test_coverage_api_docs
0.55s call     tests/test_project.py::test_doctest_api_docs
0.18s call     tests/test_project.py::test_coverage
0.12s call     tests/test_project.py::test_pytest
0.09s teardown tests/test_project.py::test_tox
0.09s call     tests/test_project.py::test_subpackage
0.08s call     tests/test_project.py::test_project_folder
0.08s call     tests/test_values.py::test_double_quotes_in_name_and_description
0.07s call     tests/test_project.py::test_isort_check
0.06s call     tests/test_values.py::test_dash_in_directory_name
0.06s call     tests/test_values.py::test_space_in_directory_name
0.06s call     tests/test_values.py::test_single_quotes_in_name_and_description
0.05s call     tests/test_project.py::test_bumpversion

and after (25 seconds):

9.81s setup    tests/test_project.py::test_pytest
9.26s call     tests/test_project.py::test_tox
3.50s call     tests/test_project.py::test_prospector
0.69s call     tests/test_project.py::test_generate_api_docs
0.55s call     tests/test_project.py::test_doctest_api_docs
0.54s call     tests/test_project.py::test_coverage_api_docs
0.21s call     tests/test_project.py::test_coverage
0.12s call     tests/test_project.py::test_pytest
0.10s call     tests/test_project.py::test_subpackage
0.08s call     tests/test_project.py::test_project_folder
0.07s call     tests/test_values.py::test_dash_in_directory_name
0.06s call     tests/test_project.py::test_isort_check
0.06s call     tests/test_values.py::test_double_quotes_in_name_and_description
0.06s call     tests/test_values.py::test_space_in_directory_name
0.06s call     tests/test_values.py::test_single_quotes_in_name_and_description
0.04s call     tests/test_project.py::test_bumpversion
egpbos commented 9 months ago

Awesome! Will merge this after #347. The doc test was also fixed in that PR, so let's take that fix out of this PR.

egpbos commented 9 months ago

I rebased on main and removed the test fixes from the second commit. That left only the deprecated .project replacement, so I renamed the commit to reflect that. This resolves the merge conflicts.

egpbos commented 9 months ago

Ok, fun! Two errors detected :)

  1. Since we now generate one folder from the different tests and ruff runs on the combined end result of the other tests, which includes some generated sub-modules that don't pass ruff, it now fails on those. Solution: remove the __init__.py files from those sub-modules (ruff complains about them because they don't end in a newline), since with modern namespace sub-modules they are not even necessary anymore.
  2. The test fix from main doesn't work either anymore, because the ASCII table that is generated now with those added sub-modules does not match the ones that we got before. We should modify those expected lines. The test does appear to be a bit fragile this way, so maybe including in those lines the variable number of spaces would be better. But for now, let's keep it simple.
egpbos commented 9 months ago

It seems like __init__.py is still necessary at this point, possibly the configuration isn't yet properly set for new style namespace packages. Anyway, no problem, fix coming.

egpbos commented 9 months ago

Did a fixup rebase of the first commit to fix the tests there.

egpbos commented 9 months ago

Thanks a lot for this!

bouweandela commented 9 months ago

Thanks for fixing the merge conflict @egpbos!