deathbeds / importnb

notebook files as source
https://importnb.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
68 stars 5 forks source link

Update testing strategies for CI performance and robustness #154

Open bollwyvl opened 1 year ago

bollwyvl commented 1 year ago

Elevator Pitch

Run the tests in parallel with pytest-xdist.

Motivation

In addition to being faster, pytest-xdist can find situations where file-based things (e.g. *.pyc) can create issues when run concurrently, which do occur in the wild. It works fine with pytest-cov, as well.

To make this work, the tests that use the ref and package ifxtures in test_importnb need some work to be more hygienic.

Another CI test speed thing is to skip coverage on pypy: it's a lot slower, and can cause spurious issues (especially on windows) that wouldn't impact real-world behavior where speed is the only goal.

In CI that would just be a conditional step based on python-version, potentially moving some of the configuration arguments out of the pytest options, and into the specific invocation in the hatch jobs.