sourcefrog / cargo-mutants

:zombie: Inject bugs and see if your tests catch them!
https://mutants.rs/
MIT License
597 stars 27 forks source link

Exclude based on regex of statements? #301

Open sourcefrog opened 8 months ago

sourcefrog commented 8 months ago

https://github.com/sourcefrog/cargo-mutants/blob/b2a4bd1934d1e599afd7388791eab8ee1c839397/src/span.rs#L103

The String::with_capacity isn't wrong, and it's probably slightly beneficial to have there, but the side effects are subtle and it's probably asking too much to test it. (Actually, in this specific case we could debug_assert the capacity to shush it, but that's a bit messy.)

We cannot yet add a #[mutants::skip] here, because expression attributes aren't stable yet (https://github.com/rust-lang/rust/issues/15701).

Maybe it would be useful to have config that can match against the whole expression text and then skip; that would probably help in other cases of code that only has unimportant side effects such as trace statements.

mathstuf commented 2 weeks ago

In the case you mention, isn't this at least possible?

let mut r = {
    #[cfg_attr(test, mutants::skip)]
    {
        String::with_capacity(s.len() + replacement.len());
    }
};

I do have a related request though. cargo-mutate reports that a testing utility function I have doesn't affect anything when mutated from a series of assert! macros into (). Maybe it would be possible to ignore this mutation if all of the statements are assert!-family macros (though itertools::assert_equal is in the same boat, just not an official macro).

sourcefrog commented 1 week ago

Yes, I think that framing

In the case you mention, isn't this at least possible?

let mut r = {
    #[cfg_attr(test, mutants::skip)]
    {
        String::with_capacity(s.len() + replacement.len());
    }
};

The stable compiler also tells me that's not supported yet, although you could do it on nightly:

error[E0658]: attributes on expressions are experimental
  --> tests/in_diff.rs:17:26
   |
17 |         #[cfg_attr(test, mutants::skip)]
   |                          ^^^^^^^^^^^^^
   |
   = note: see issue #15701 <https://github.com/rust-lang/rust/issues/15701> for more information
error[E0658]: custom attributes cannot be applied to expressions
  --> tests/in_diff.rs:17:26
   |
17 |         #[cfg_attr(test, mutants::skip)]
   |                          ^^^^^^^^^^^^^
   |
   = note: see issue #54727 <https://github.com/rust-lang/rust/issues/54727> for more information

Skipping functions entirely full of assertions seems like a good idea, probably. Although in that case you really could just mark it as #[mutants::skip].