boxed / mutmut

Mutation testing system
https://mutmut.readthedocs.io
BSD 3-Clause "New" or "Revised" License
932 stars 112 forks source link

Invalid mutations involving positional-only and keyword-only arguments #303

Closed qthequartermasterman closed 1 month ago

qthequartermasterman commented 9 months ago

I have an odd false-positive surviving mutant (when using the standard pytest runner) involving positional-only and keyword-only arguments.

Consider this function, with two positional-only arguments:

def foo(a, b, /, **kwargs):
    print(a, b, kwargs)

foo(1, 2, c=3)  # 1 2 {'c': 3}

mutmut tried one mutation which is syntactically invalid (but somehow was still marked as survived???)

def foo_mutated(a, b, *, **kwargs):  # SyntaxError
    print(a, b, kwargs)

Ideally, mutmut wouldn't generate this mutation because it's a SyntaxError.

However, it is only a syntax error if the * and **kwargs are adjacent (i.e. no explicitly named parameters). Consider this:

def foo_with_c(a, b, /, c, **kwargs):
    print(a, b, c, kwargs)

foo_with_c(1, 2, c=3)  # 1 2 3 {}

def foo_mutated_with_c(a, b, *, c, **kwargs):
    print(a, b, c, kwargs)

foo_mutated_with_c(1, 2, c=3)  # 1 2 3 {}

This mutation is syntactically valid, and thus should be tried. Consequently, simply avoiding replacing / with * in function signatures is not sufficient.

If there is not an obvious way to only generate the syntactically correct mutants, is there a way to guarantee that mutants with syntax errors are not marked as survived?

boxed commented 9 months ago

Yea I guess we should load the resulting file with the ast standard library and if it's invalid just mark it as killed. This should be super fast and just a clean win.

boxed commented 1 month ago

I just released mutmut 3, which is a big rewrite.

The fix I suggested here is implemented in that new code.