search

boxed / mutmut

Mutation testing system
https://mutmut.readthedocs.io
BSD 3-Clause "New" or "Revised" License
902 stars 109 forks source link

Strange behaviour with multiple context managers. #329

Closed EdgyEdgemond closed 3 months ago

EdgyEdgemond commented 3 months ago

Using multiple context managers in a single with (recommended by linters) breaks the AST parser. Removing the second context manager and just doing with foo() as f: print(f) parses happily.

from contextlib import contextmanager

@contextmanager
async def foo():
    yield "foo"

@contextmanager
def bar():
    yield "bar"

def func():
    with (
        foo() as f,
        bar(),
    )
        print(f)
Failed to parse code.py. Internal error from parso follows.
----------------------------------
Traceback (most recent call last):
  File "/scripts/.venv/lib/python3.11/site-packages/parso/parser.py", line 180, in _add_token
    plan = stack[-1].dfa.transitions[transition]
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: ReservedString(as)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__init__.py", line 1244, in add_mutations_by_file
    mutations_by_file[filename] = list_mutations(context)
                                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__init__.py", line 686, in list_mutations
    mutate(context)
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__init__.py", line 571, in mutate
    result = parse(context.source, error_recovery=False)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/parso/__init__.py", line 58, in parse
    return grammar.parse(code, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/parso/grammar.py", line 156, in parse
    root_node = p.parse(tokens=tokens)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/parso/python/parser.py", line 79, in parse
    return super().parse(tokens)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/parso/parser.py", line 129, in parse
    self._add_token(token)
  File "/scripts/.venv/lib/python3.11/site-packages/parso/parser.py", line 186, in _add_token
    self.error_recovery(token)
  File "/scripts/.venv/lib/python3.11/site-packages/parso/python/parser.py", line 140, in error_recovery
    return super().error_recovery(token)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/parso/parser.py", line 152, in error_recovery
    raise ParserSyntaxError('SyntaxError: invalid syntax', error_leaf)
parso.parser.ParserSyntaxError: ('SyntaxError: invalid syntax', <ErrorLeaf: PythonTokenTypes.NAME:'as', (16, 14)>)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/scripts/.venv/bin/mutmut", line 8, in <module>
    sys.exit(climain())
             ^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__init__.py", line 893, in wrapper
    f(*args, **kwargs)
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__main__.py", line 170, in run
    sys.exit(do_run(argument, paths_to_mutate, disable_mutation_types, enable_mutation_types, runner,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__main__.py", line 448, in do_run
    parse_run_argument(argument, config, dict_synonyms, mutations_by_file, paths_to_exclude, paths_to_mutate, tests_dirs)
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__main__.py", line 476, in parse_run_argument
    add_mutations_by_file(mutations_by_file, filename, dict_synonyms, config)
  File "/scripts/.venv/lib/python3.11/site-packages/mutmut/__init__.py", line 1249, in add_mutations_by_file
    raise RuntimeError(
RuntimeError: Failed while creating mutations for code.py, for line "from contextlib import contextmanager"
EdgyEdgemond commented 3 months ago

Underlying parso issue, already raised over there...

https://github.com/davidhalter/parso/issues/203

boxed commented 3 months ago

Yea. We probably need to switch AST lib