Tests fail if not 'codespell' installed

[anatol]

It looks like the project tests use codespell from $PATH rather than from the source tree.

I am following Arch linux build instructions. I build the project with python -m build --wheel --no-isolation then run tests with pytest and it fails:

$ pytest
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.12.3, pytest-8.2.1, pluggy-1.5.0
rootdir: /storage/sources/codespell
configfile: pyproject.toml
testpaths: codespell_lib/tests
plugins: dependency-0.6.0, cov-4.1.0, typeguard-4.2.1
collected 113 items                                                                                                                                                                                               

codespell_lib/tests/test_basic.py .F........................................................F                                                                                                               [ 52%]
codespell_lib/tests/test_dictionary.py ..............................ssssssssssssssss........                                                                                                               [100%]

==================================================================================================== FAILURES =====================================================================================================
__________________________________________________________________________________________________ test_command ___________________________________________________________________________________________________
codespell_lib/tests/test_basic.py:85: in test_command
    assert run_codespell(cwd=tmp_path) == 0
codespell_lib/tests/test_basic.py:72: in run_codespell
    proc = subprocess.run(
/usr/lib/python3.12/subprocess.py:548: in run
    with Popen(*popenargs, **kwargs) as process:
/usr/lib/python3.12/subprocess.py:1026: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
/usr/lib/python3.12/subprocess.py:1955: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   FileNotFoundError: [Errno 2] No such file or directory: 'codespell'
___________________________________________________________________________________________________ test_stdin ____________________________________________________________________________________________________
codespell_lib/tests/test_basic.py:1367: in test_stdin
    assert run_codespell_stdin(
codespell_lib/tests/test_basic.py:1342: in run_codespell_stdin
    proc = subprocess.run(
/usr/lib/python3.12/subprocess.py:548: in run
    with Popen(*popenargs, **kwargs) as process:
/usr/lib/python3.12/subprocess.py:1026: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
/usr/lib/python3.12/subprocess.py:1955: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   FileNotFoundError: [Errno 2] No such file or directory: 'codespell'
------------------------------------------------------------------------ generated xml file: /storage/sources/codespell/junit-results.xml -------------------------------------------------------------------------
============================================================================================= short test summary info =============================================================================================
SKIPPED [16] codespell_lib/tests/test_dictionary.py:223: requires aspell-en
==================================================================================== 2 failed, 95 passed, 16 skipped in 56.63s ====================================================================================

[DimitriPapadopoulos]

I was unable to find evidence that tests run codespell from $PATH in the logs you're provide, but you're probably right: https://github.com/codespell-project/codespell/blob/8097264ed0fd207777c001dcf544beec14e36155/codespell_lib/tests/test_basic.py#L66-L73

Perhaps we expect codespell to be installed in dev mode before running the tests.

[anatol]

Perhaps we expect codespell to be installed in dev mode before running the tests.

This won't work in all cases. e.g. many Linux distros use a clean build environment with precise set of dependencies. codespell cannot be installed there as otherwise it will be a chicken-egg problem.

Instead expecting codespell installed, the tests should use path to newly built binary, or add that directory to $PATH.

[DimitriPapadopoulos]

It makes sense. However, the Development setup does not provide a "newly built binary", apart from the installed one. How to combine both?

[DimitriPapadopoulos]

See also #2595.

@anatol What do you mean exactly by "newly built binary"? Note that the codespell script is created by setuptools: https://github.com/codespell-project/codespell/blob/523b9c57b58a9a857e9e1dc44dabe3455cc4867d/pyproject.toml#L60-L61 I understand that setuptools creates the script while installing. If so, it makes sense to install (perhaps in a temporary location) before testing.

[DimitriPapadopoulos]

On my workstation, the generated codespell scripts looks like this:

#!/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from codespell_lib import _script_main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(_script_main())

We could of course create and run something similar in our tests instead of running codespell. But what's wrong with testing the generated script (even within a temporary location) instead of testing our idea of what the generated script looks like?

Right now, we haven't heard from Linux manitainers, and therefore haven't got actual feedback about the chicken-egg problem, or how to circumvent it.