search

sphinx-contrib / spelling

A spelling checker for Sphinx-based documentation
https://sphinxcontrib-spelling.readthedocs.io/en/latest/
BSD 2-Clause "Simplified" License
82 stars 41 forks source link

8.0.0: pytest is failing #215

Open kloczek opened 1 year ago

kloczek commented 1 year ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-spelling-8.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-spelling-8.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.16, pytest-7.3.1, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/spelling-8.0.0 collected 6 items / 2 errors ========================================================================================== ERRORS =========================================================================================== __________________________________________________________________________ ERROR collecting tests/test_builder.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/spelling-8.0.0/tests/test_builder.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib64/python3.8/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/test_builder.py:16: in from tests import helpers # isort:skip E ModuleNotFoundError: No module named 'tests' ___________________________________________________________________________ ERROR collecting tests/test_filter.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/spelling-8.0.0/tests/test_filter.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib64/python3.8/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/test_filter.py:16: in from tests import helpers # isort:skip E ModuleNotFoundError: No module named 'tests' ================================================================================== short test summary info ================================================================================== ERROR tests/test_builder.py ERROR tests/test_filter.py !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ===================================================================================== 2 errors in 0.35s ===================================================================================== ```

Here is list of installed modules in build env

```console Package Version ----------------------------- ----------------- alabaster 0.7.13 Babel 2.12.1 build 0.10.0 charset-normalizer 3.1.0 distro 1.8.0 docutils 0.19 exceptiongroup 1.0.0 fixtures 4.0.1 gpg 1.19.0 idna 3.4 imagesize 1.4.1 importlib-metadata 6.5.0 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.2 libcomps 0.1.19 MarkupSafe 2.1.2 packaging 23.0 pbr 5.11.1 pluggy 1.0.0 pyenchant 3.2.2 Pygments 2.15.1 pyproject_hooks 1.0.0 pytest 7.3.1 python-dateutil 2.8.2 pytz 2023.2 requests 2.28.2 setuptools 67.6.1 setuptools-scm 7.1.0 six 1.16.0 snowballstemmer 2.2.0 Sphinx 6.1.3 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2.dev20230415 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1.dev20230415 sphinxcontrib-qthelp 1.0.3.dev20230415 sphinxcontrib-serializinghtml 1.1.5 sphinxcontrib-spelling 7.7.0 tomli 2.0.1 typing_extensions 4.5.0 urllib3 1.26.15 wheel 0.40.0 zipp 3.15.0 ```
kloczek commented 7 months ago

gentle ping 🤔

dhellmann commented 7 months ago

What are the contents of the directory where you're running the tests.

The tox target isolated demonstrates how to run the tests outside of the source code directory.

kloczek commented 6 months ago

I'm executing pytest in source root directory. Looks like tests/init.py is missing. After create empty tests/init.py pytest is able to pass units scanning however it fails in some units

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-spelling-8.0.0-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-spelling-8.0.0-2.fc36.x86_64/usr/lib/python3.9/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0 rootdir: /home/tkloczko/rpmbuild/BUILD/spelling-8.0.0 configfile: pyproject.toml plugins: mock-3.12.0 collected 67 items tests/test_builder.py ...F......s....FFFF.FF.. [ 35%] tests/test_checker.py FFFF.. [ 44%] tests/test_filter.py ....sssssssssssssssssssssssssss...... [100%] ========================================================================================= FAILURES ========================================================================================== ___________________________________________________________________________________ test_ignore_literals ____________________________________________________________________________________ sphinx_project = (local('/tmp/pytest-of-tkloczko/pytest-66/test_ignore_literals0/src'), local('/tmp/pytest-of-tkloczko/pytest-66/test_ignore_literals0/out')) def test_ignore_literals(sphinx_project): srcdir, outdir = sphinx_project add_file(srcdir, 'contents.rst', ''' Welcome to Spelling Checker documentation! ========================================== There are several misspelled words in this text. :: Literal blocks are ignoreed. Inline ``litterals`` are ignored, too. ''') stdout, stderr, output_text = get_sphinx_output(srcdir, outdir, 'contents') # The 'contents.spelling' output file should not have been # created, because the errors are ignored. > assert output_text is None E AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_ignore_literals0/src/contents.rst:3: (Welcome) Welcome to Spell...../../../../../tmp/pytest-of-tkloczko/pytest-66/test_ignore_literals0/src/contents.rst:11: (too) are ignored, too.\n' is None tests/test_builder.py:138: AssertionError ----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------ fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------- WARNING test.filters:filters.py:249 Called: git log --quiet --no-color --pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn WARNING test.filters:filters.py:250 Failed to scan contributors: Command '['git', 'log', '--quiet', '--no-color', '--pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn']' returned non-zero exit status 128. ___________________________________________________________________________________ test_legacy_directive ___________________________________________________________________________________ sphinx_project = (local('/tmp/pytest-of-tkloczko/pytest-66/test_legacy_directive0/src'), local('/tmp/pytest-of-tkloczko/pytest-66/test_legacy_directive0/out')) def test_legacy_directive(sphinx_project): srcdir, outdir = sphinx_project add_file(srcdir, 'contents.rst', ''' The Module ========== .. spelling:: teh teh is OK ''') stdout, stderr, output_text = get_sphinx_output( srcdir, outdir, 'contents', ) > assert output_text is None E AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_legacy_directive0/src/contents.rst:3: (The) The Module\n../../....e Module\n../../../../../tmp/pytest-of-tkloczko/pytest-66/test_legacy_directive0/src/contents.rst:9: (is) teh is OK\n' is None tests/test_builder.py:386: AssertionError ----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------ fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------- WARNING test.filters:filters.py:249 Called: git log --quiet --no-color --pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn WARNING test.filters:filters.py:250 Failed to scan contributors: Command '['git', 'log', '--quiet', '--no-color', '--pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn']' returned non-zero exit status 128. ___________________________________________________________________________________ test_domain_directive ___________________________________________________________________________________ sphinx_project = (local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_directive0/src'), local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_directive0/out')) def test_domain_directive(sphinx_project): srcdir, outdir = sphinx_project add_file(srcdir, 'contents.rst', ''' The Module ========== .. spelling:word-list:: teh teh is OK ''') stdout, stderr, output_text = get_sphinx_output( srcdir, outdir, 'contents', ) > assert output_text is None E AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_directive0/src/contents.rst:3: (The) The Module\n../../....e Module\n../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_directive0/src/contents.rst:9: (is) teh is OK\n' is None tests/test_builder.py:409: AssertionError ----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------ fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------- WARNING test.filters:filters.py:249 Called: git log --quiet --no-color --pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn WARNING test.filters:filters.py:250 Failed to scan contributors: Command '['git', 'log', '--quiet', '--no-color', '--pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn']' returned non-zero exit status 128. _____________________________________________________________________________________ test_domain_role ______________________________________________________________________________________ sphinx_project = (local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_role0/src'), local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_role0/out')) def test_domain_role(sphinx_project): srcdir, outdir = sphinx_project add_file(srcdir, 'contents.rst', ''' The Module ========== :spelling:word:`teh` is OK ''') stdout, stderr, output_text = get_sphinx_output( srcdir, outdir, 'contents', ) > assert output_text is None E AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_role0/src/contents.rst:3: (The) The Module\n../../../../...ule) The Module\n../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_role0/src/contents.rst:5: (is) is OK\n' is None tests/test_builder.py:428: AssertionError ----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------ fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------- WARNING test.filters:filters.py:249 Called: git log --quiet --no-color --pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn WARNING test.filters:filters.py:250 Failed to scan contributors: Command '['git', 'log', '--quiet', '--no-color', '--pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn']' returned non-zero exit status 128. ______________________________________________________________________________ test_domain_role_multiple_words ______________________________________________________________________________ sphinx_project = (local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_role_multiple_word0/src'), local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_role_multiple_word0/out')) def test_domain_role_multiple_words(sphinx_project): srcdir, outdir = sphinx_project add_file(srcdir, 'contents.rst', ''' The Module ========== :spelling:word:`teh is KO` ''') stdout, stderr, output_text = get_sphinx_output( srcdir, outdir, 'contents', ) > assert output_text is None E AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_role_multiple_word0/src/contents.rst:3: (The) The Module..../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_role_multiple_word0/src/contents.rst:3: (Module) The Module\n' is None tests/test_builder.py:447: AssertionError ----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------ fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------- WARNING test.filters:filters.py:249 Called: git log --quiet --no-color --pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn WARNING test.filters:filters.py:250 Failed to scan contributors: Command '['git', 'log', '--quiet', '--no-color', '--pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn']' returned non-zero exit status 128. ____________________________________________________________________________________ test_domain_ignore _____________________________________________________________________________________ sphinx_project = (local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore0/src'), local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore0/out')) def test_domain_ignore(sphinx_project): srcdir, outdir = sphinx_project add_file(srcdir, 'contents.rst', ''' The Module ========== :spelling:ignore:`baddddd` is OK ''') stdout, stderr, output_text = get_sphinx_output( srcdir, outdir, 'contents', ) > assert output_text is None E AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore0/src/contents.rst:3: (The) The Module\n../../../....e) The Module\n../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore0/src/contents.rst:5: (is) is OK\n' is None tests/test_builder.py:494: AssertionError ----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------ fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------- WARNING test.filters:filters.py:249 Called: git log --quiet --no-color --pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn WARNING test.filters:filters.py:250 Failed to scan contributors: Command '['git', 'log', '--quiet', '--no-color', '--pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn']' returned non-zero exit status 128. _____________________________________________________________________________ test_domain_ignore_multiple_words _____________________________________________________________________________ sphinx_project = (local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore_multiple_wo0/src'), local('/tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore_multiple_wo0/out')) def test_domain_ignore_multiple_words(sphinx_project): srcdir, outdir = sphinx_project add_file(srcdir, 'contents.rst', ''' The Module ========== :spelling:ignore:`baddddd` is OK here. But, baddddd is not OK here. Nor, here baddddd. ''') stdout, stderr, output_text = get_sphinx_output( srcdir, outdir, 'contents', ) assert '(baddddd)' in output_text > assert output_text.count('\n') == 2 # Only expect 2 errors, not 3. E AssertionError: assert 12 == 2 E + where 12 = ('\n') E + where = '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore_multiple_wo0/src/contents.rst:3: (The) The Module..../tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore_multiple_wo0/src/contents.rst:8: (baddddd) Nor, here baddddd.\n'.count tests/test_builder.py:517: AssertionError ----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------ fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------- WARNING test.filters:filters.py:249 Called: git log --quiet --no-color --pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn WARNING test.filters:filters.py:250 Failed to scan contributors: Command '['git', 'log', '--quiet', '--no-color', '--pretty=format:%(trailers:key=Co-Authored-By,separator=%x0A)%x0A%an%x0A%cn']' returned non-zero exit status 128. _____________________________________________________________________________________ test_errors_only ______________________________________________________________________________________ def test_errors_only(): checker = SpellingChecker(lang='en_US', suggest=False, word_list_filename=None, ) for word, suggestions, line, offset in checker.check('This txt is wrong'): assert not suggestions, 'Suggesting' > assert word == 'txt' E AssertionError: assert 'This' == 'txt' E E - txt E + This tests/test_checker.py:19: AssertionError ___________________________________________________________________________________ test_with_suggestions ___________________________________________________________________________________ def test_with_suggestions(): checker = SpellingChecker(lang='en_US', suggest=True, word_list_filename=None, ) for word, suggestions, line, offset in checker.check('This txt is wrong'): > assert suggestions, 'Not suggesting' E AssertionError: Not suggesting E assert [] tests/test_checker.py:30: AssertionError ____________________________________________________________________________________ test_with_wordlist _____________________________________________________________________________________ def test_with_wordlist(): checker = SpellingChecker( lang='en_US', suggest=False, word_list_filename=os.path.join(os.path.dirname(__file__), 'test_wordlist.txt') ) words = [w for w, s, l, o in checker.check('This txt is wrong')] > assert not words, 'Did not use personal word list file' E AssertionError: Did not use personal word list file E assert not ['This', 'is', 'wrong'] tests/test_checker.py:43: AssertionError __________________________________________________________________________________ test_with_context_line ___________________________________________________________________________________ def test_with_context_line(): checker = SpellingChecker(lang='en_US', suggest=False, word_list_filename=None, context_line=True, ) text = 'Line one\nThis txt is wrong\nLine two' for word, suggestions, line, offset in checker.check(text): assert not suggestions, 'Suggesting' > assert word == 'txt' E AssertionError: assert 'Line' == 'txt' E E - txt E + Line tests/test_checker.py:56: AssertionError ================================================================================== short test summary info ================================================================================== SKIPPED [1] tests/test_builder.py:257: Not a git repo SKIPPED [27] tests/test_filter.py:49: Not a git repo FAILED tests/test_builder.py::test_ignore_literals - AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_ignore_literals0/src/contents.rst:3: (Welcome) Welcome to Spell...../../../../../tmp/pytest-of-tkloczko/py... FAILED tests/test_builder.py::test_legacy_directive - AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_legacy_directive0/src/contents.rst:3: (The) The Module\n../../....e Module\n../../../../../tmp/pytest-of-t... FAILED tests/test_builder.py::test_domain_directive - AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_directive0/src/contents.rst:3: (The) The Module\n../../....e Module\n../../../../../tmp/pytest-of-t... FAILED tests/test_builder.py::test_domain_role - AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_role0/src/contents.rst:3: (The) The Module\n../../../../...ule) The Module\n../../../../../tmp/pyt... FAILED tests/test_builder.py::test_domain_role_multiple_words - AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_role_multiple_word0/src/contents.rst:3: (The) The Module..../../../../tmp/pytest-of-tkloczko/pytest... FAILED tests/test_builder.py::test_domain_ignore - AssertionError: assert '../../../../../tmp/pytest-of-tkloczko/pytest-66/test_domain_ignore0/src/contents.rst:3: (The) The Module\n../../../....e) The Module\n../../../../../tmp/pytes... FAILED tests/test_builder.py::test_domain_ignore_multiple_words - AssertionError: assert 12 == 2 FAILED tests/test_checker.py::test_errors_only - AssertionError: assert 'This' == 'txt' FAILED tests/test_checker.py::test_with_suggestions - AssertionError: Not suggesting FAILED tests/test_checker.py::test_with_wordlist - AssertionError: Did not use personal word list file FAILED tests/test_checker.py::test_with_context_line - AssertionError: assert 'Line' == 'txt' ========================================================================= 11 failed, 28 passed, 28 skipped in 3.79s ========================================================================= ```
dhellmann commented 6 months ago
  • PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-spelling-8.0.0-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-spelling-8.0.0-2.fc36.x86_64/usr/lib/python3.9/site-packages
  • /usr/bin/pytest -ra -m 'not network'

I'm not sure what you're doing there with the python path. It's possible you don't have enough entries for the interpreter to find everything. If you look at what the isolated target in tox.ini does, you will see that it copies the test files out of the tests directory into a temporary directory and then runs pytest there, rather than from the root of the source tree.

kloczek commented 6 months ago

I'm not sure what you're doing there with the python path. It's possible you don't have enough entries for the interpreter to find everything.

That is only entry added on top of default content of the sys.path. Executing test suite in in tox does not make to much sense in my case as package build is executed inside dedicated build env created only to build that package and in which are installed only build dependencies.

-m 'not network' is standard part of my %pytest rpm macro (many python modules already standardised around that pytest mark to disable execution of the units which needs access to more than localhost). In this case it is not relevant,

If you look at what the isolated target in tox.ini does, you will see that it copies the test files out of the tests directory into a temporary directory and then runs pytest there, rather than from the root of the source tree.

None of the test suites should rely on such special procedures like copy something to special directory. Try to think that some build infras are building sometimes packages out of R/O directory structure. Please read about some examples of directories layouts provided in pytest documentation https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html

Did you try to reproduce this issue? 🤔