nox test session is too slow for unit testing

[jreiberkyle]

Even with a re-used test environment, it takes 20 seconds to perform a unit test run with time nox -r -s test -p 3.7. This disrupts test-driven development.

There is some extra / custom code for the test session in particular that is slowing it down:

    session.run('python', '-m', 'ensurepip', '--upgrade')
    session.install('-U', 'setuptools')
    session.install(".[test]")

Removing the upgrade/install and running the editable version of the code cuts the time for a unit test run to 0.5 seconds.

Workaround

Make a custom noxfile with the speed-up changes, aka mynoxfile.py and then run nox -r -f mynoxfile.py

Results

As is:

> time nox -s test -p 3.7
real    0m48.848s
user    0m21.323s
sys 0m20.771s

> time nox -r -s test -p 3.7
real    0m40.590s
user    0m15.247s
sys 0m20.071s

If we rewrite the nox test session from

@nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"])
def test(session):
    session.run('python', '-m', 'ensurepip', '--upgrade')
    session.install('-U', 'setuptools')
    session.install(".[test]")

    options = session.posargs
    # -W=error raises pytest warnings to errors so they are caught by CI
    # to exclude some warnings, see
    # https://docs.python.org/3/library/warnings.html#temporarily-suppressing-warnings
    session.run('python',
                '-m',
                'pytest',
                '--ignore',
                'examples/',
                '-v',
                '-Werror',
                '-Wignore::DeprecationWarning:tqdm.std',
                *options)

to

@nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"])
def test(session):
    # NOTE: removed pip upgrade and setuptools install as they slowed down the
    # session when reusing an environment considerably
    # NOTE: using 'e' here speeds up the session considerably
    session.install("-e", ".[test]")

    options = session.posargs
    # -W=error raises pytest warnings to errors so they are caught by CI
    # to exclude some warnings, see
    # https://docs.python.org/3/library/warnings.html#temporarily-suppressing-warnings
    session.run('python',
                '-m',
                'pytest',
                '--ignore',
                'examples/',
                '-v',
                '-Werror',
                '-Wignore::DeprecationWarning:tqdm.std',
                *options)

we get

> time nox -r -s test -p 3.7
real    0m6.071s
user    0m4.977s
sys 0m0.516s