search

boxed / mutmut

Mutation testing system
https://mutmut.readthedocs.io
BSD 3-Clause "New" or "Revised" License
931 stars 112 forks source link

Various errors when running mutmut #343

Open azmeuk opened 1 week ago

azmeuk commented 1 week ago

I wanted to try mutmut against scim2-models. My setup.cfg file looks like this:

[mutmut]
paths_to_mutate=scim2_models/
tests_dir=tests/
also_copy=
    conftest.py

When I run mutmut (either 3.2.1 or from the main branch here) inside the project venv I get this errors:

Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/_virtualenv.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named '_virtualenv'

Remainder of file ignored
Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/pre_commit_uv_patch.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  AttributeError: module 'pre_commit_uv' has no attribute '_patch'

Remainder of file ignored
Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/_virtualenv.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named '_virtualenv'

Remainder of file ignored
Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/pre_commit_uv_patch.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  AttributeError: module 'pre_commit_uv' has no attribute '_patch'

Remainder of file ignored
Usage: mutmut [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  apply
  browse
  print-time-estimates
  results
  run
  show
  tests-for-mutant

And mutmut run gives me this:

mutmut run
Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/_virtualenv.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named '_virtualenv'

Remainder of file ignored
Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/pre_commit_uv_patch.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  AttributeError: module 'pre_commit_uv' has no attribute '_patch'

Remainder of file ignored
Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/_virtualenv.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named '_virtualenv'

Remainder of file ignored
Error processing line 1 of /home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/pre_commit_uv_patch.pth:

  Traceback (most recent call last):
    File "<frozen site>", line 206, in addpackage
    File "<string>", line 1, in <module>
  AttributeError: module 'pre_commit_uv' has no attribute '_patch'

Remainder of file ignored
⠋ Generating mutants
    done in 3ms
⠙ Running stats
Traceback (most recent call last):
  File "/home/eloi/dev/python-scim/scim2-models/.venv/bin/mutmut", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/mutmut/__main__.py", line 1238, in run
    collect_or_load_stats(runner)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/mutmut/__main__.py", line 1064, in collect_or_load_stats
    run_stats_collection(runner)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/mutmut/__main__.py", line 1042, in run_stats_collection
    collect_stats_exit_code = runner.run_stats(tests=tests)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/mutmut/__main__.py", line 724, in run_stats
    return int(self.execute_pytest(['-x', '-q', '--import-mode=append'] + list(tests), plugins=[stats_collector]))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.12/site-packages/mutmut/__main__.py", line 701, in execute_pytest
    exit_code = int(pytest.main(params, **kwargs))
                    ^^^^^^^^^^^
AttributeError: module 'pytest' has no attribute 'main'

I run Archlinux with Python 3.12, and here is the output of uv pip freeze:

annotated-types==0.7.0
cachetools==5.5.0
cfgv==3.4.0
chardet==5.2.0
click==8.1.7
colorama==0.4.6
coverage==7.6.4
distlib==0.3.9
dnspython==2.7.0
email-validator==2.2.0
filelock==3.16.1
identify==2.6.1
idna==3.10
iniconfig==2.0.0
junit-xml==1.8
linkify-it-py==2.0.3
markdown-it-py==3.0.0
mdit-py-plugins==0.4.2
mdurl==0.1.2
mutmut @ git+https://github.com/boxed/mutmut.git@cb78ed2e666bcc2778ab38352be013fbe1028e10
mypy==1.13.0
mypy-extensions==1.0.0
nodeenv==1.9.1
packaging==24.1
parso==0.8.4
platformdirs==4.3.6
pluggy==1.5.0
pre-commit==4.0.1
pre-commit-uv==4.1.4
pydantic==2.9.2
pydantic-core==2.23.4
pygments==2.18.0
pyproject-api==1.8.0
pytest==8.3.3
pytest-cov==6.0.0
pyyaml==6.0.2
rich==13.9.4
-e file:///home/eloi/dev/python-scim/scim2-models
setproctitle==1.3.3
six==1.16.0
textual==0.85.2
tox==4.23.2
tox-uv==1.16.0
typing-extensions==4.12.2
uc-micro-py==1.0.3
uv==0.4.29
virtualenv==20.27.1

If I run the same project with Python 3.9 instead I have different errors:

$ uv venv --python 3.9
$ uv sync
$ uv pip install mutmut
$ mutmut
Usage: mutmut [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  apply
  browse
  print-time-estimates
  results
  run
  show
  tests-for-mutant
$ mutmut run
⠋ Generating mutants
    done in 4ms
⠙ Running stats
Traceback (most recent call last):
  File "/home/eloi/dev/python-scim/scim2-models/.venv/bin/mutmut", line 8, in <module>
    sys.exit(cli())
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/mutmut/__main__.py", line 1238, in run
    collect_or_load_stats(runner)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/mutmut/__main__.py", line 1064, in collect_or_load_stats
    run_stats_collection(runner)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/mutmut/__main__.py", line 1042, in run_stats_collection
    collect_stats_exit_code = runner.run_stats(tests=tests)
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/mutmut/__main__.py", line 724, in run_stats
    return int(self.execute_pytest(['-x', '-q', '--import-mode=append'] + list(tests), plugins=[stats_collector]))
  File "/home/eloi/dev/python-scim/scim2-models/.venv/lib/python3.9/site-packages/mutmut/__main__.py", line 705, in execute_pytest
    raise BadTestExecutionCommandsException(params)
mutmut.__main__.BadTestExecutionCommandsException: ['--doctest-modules', '--doctest-glob=*.rst', '-x', '-q', '--import-mode=append', '--rootdir=.']

This error looks a lot like the one in #340.

If I run this command by hand at the project root it gives me this:

$ pytest --doctest-modules '--doctest-glob=*.rst' -x -q --import-mode=append --rootdir=.

====================================================== ERRORS ======================================================
__________________________________ ERROR collecting mutants/scim2_models/base.py ___________________________________
import file mismatch:
imported module 'scim2_models.base' has this __file__ attribute:
  /home/eloi/dev/python-scim/scim2-models/scim2_models/base.py
which is not the same as the test file we want to collect:
  /home/eloi/dev/python-scim/scim2-models/mutants/scim2_models/base.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
============================================= short test summary info ==============================================
ERROR mutants/scim2_models/base.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1 error in 0.02s

If I run the command by hand from the mutants dir I get this:

$ pytest --doctest-modules '--doctest-glob=*.rst' -x -q --import-mode=append --rootdir=.
ImportError while loading conftest '/home/eloi/dev/python-scim/scim2-models/mutants/conftest.py'.
_pytest.pathlib.ImportPathMismatchError: ('conftest', '/home/eloi/dev/python-scim/scim2-models/conftest.py', PosixPath('/home/eloi/dev/python-scim/scim2-models/mutants/conftest.py'))

If I remove conftest.py from the root of the project, it looks like I have a new issue with pydantic:

$ mutmut run
⠼ Generating mutants
    done in 4ms
⠼ Running stats
====================================================== ERRORS ======================================================
____________________________________ ERROR collecting scim2_models/__init__.py _____________________________________
scim2_models/__init__.py:46: in <module>
    from .base import BaseModel
scim2_models/base.py:2255: in <module>
    class BaseModel(PydanticBaseModel):
../.venv/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py:115: in __new__
    private_attributes = inspect_namespace(
../.venv/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py:428: in inspect_namespace
    raise PydanticUserError(
E   pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `xǁBaseModelǁmark_with_schema__mutmut_mutants = {'xǁBaseModelǁmark_with_schema__mutmut_1': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_1 at 0x7ae5be304af0>, 'xǁBaseModelǁmark_with_schema__mutmut_2': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_2 at 0x7ae5be304b80>, 'xǁBaseModelǁmark_with_schema__mutmut_3': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_3 at 0x7ae5be304c10>, 'xǁBaseModelǁmark_with_schema__mutmut_4': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_4 at 0x7ae5be304ca0>, 'xǁBaseModelǁmark_with_schema__mutmut_5': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_5 at 0x7ae5be304d30>, 'xǁBaseModelǁmark_with_schema__mutmut_6': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_6 at 0x7ae5be304dc0>, 'xǁBaseModelǁmark_with_schema__mutmut_7': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_7 at 0x7ae5be304e50>, 'xǁBaseModelǁmark_with_schema__mutmut_8': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_8 at 0x7ae5be304ee0>, 'xǁBaseModelǁmark_with_schema__mutmut_9': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_9 at 0x7ae5be304f70>, 'xǁBaseModelǁmark_with_schema__mutmut_10': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_10 at 0x7ae5be30b040>, 'xǁBaseModelǁmark_with_schema__mutmut_11': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_11 at 0x7ae5be30b0d0>, 'xǁBaseModelǁmark_with_schema__mutmut_12': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_12 at 0x7ae5be30b160>, 'xǁBaseModelǁmark_with_schema__mutmut_13': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_13 at 0x7ae5be30b1f0>, 'xǁBaseModelǁmark_with_schema__mutmut_14': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_14 at 0x7ae5be30b280>, 'xǁBaseModelǁmark_with_schema__mutmut_15': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_15 at 0x7ae5be30b310>, 'xǁBaseModelǁmark_with_schema__mutmut_16': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_16 at 0x7ae5be30b3a0>, 'xǁBaseModelǁmark_with_schema__mutmut_17': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_17 at 0x7ae5be30b430>, 'xǁBaseModelǁmark_with_schema__mutmut_18': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_18 at 0x7ae5be30b4c0>, 'xǁBaseModelǁmark_with_schema__mutmut_19': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_19 at 0x7ae5be30b550>, 'xǁBaseModelǁmark_with_schema__mutmut_20': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_20 at 0x7ae5be30b5e0>, 'xǁBaseModelǁmark_with_schema__mutmut_21': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_21 at 0x7ae5be30b670>, 'xǁBaseModelǁmark_with_schema__mutmut_22': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_22 at 0x7ae5be30b700>, 'xǁBaseModelǁmark_with_schema__mutmut_23': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_23 at 0x7ae5be30b790>, 'xǁBaseModelǁmark_with_schema__mutmut_24': <function BaseModel.xǁBaseModelǁmark_with_schema__mutmut_24 at 0x7ae5be30b820>}`. All model fields require a type annotation; if `xǁBaseModelǁmark_with_schema__mutmut_mutants` is not meant to be a field, you may be able to resolve this error by annotating it as a `ClassVar` or updating `model_config['ignored_types']`.
E
E   For further information visit https://errors.pydantic.dev/2.9/u/model-field-missing-annotation
============================================= short test summary info ==============================================
ERROR scim2_models/__init__.py - pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `xǁBa...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1 error in 0.17s
failed to collect stats. runner returned 1
mattwthompson commented 5 days ago

I'm running into this as well. I think #302 is related