search

sourcefrog / cargo-mutants

:zombie: Inject bugs and see if your tests catch them!
https://mutants.rs/
MIT License
597 stars 27 forks source link

RFE: generate a sourcemap of what code affects which tests #427

Open mathstuf opened 2 weeks ago

mathstuf commented 2 weeks ago

It would be nice to be able to generate some kind of database that records what tests cover which source lines. This could then be consumed by something like nextest in conjunction with git diff to only run tests that are affected by the changes in a patch to avoid running the entire test suite all the time.

mathstuf commented 2 weeks ago

As a bonus, CI could also mutate just the diff and run tests in that direction too.

sourcefrog commented 1 week ago

This is similar to #24.

But, maybe here you're saying we could remember which specific tests failed due to each specific mutation?

mathstuf commented 1 week ago

But, maybe here you're saying we could remember which specific tests failed due to each specific mutation?

Yes. When discussing with colleages, they suggested doing it by coverage, but I'm not aware of any coverage tools that save such information in a per-test bucket (especially in-process testing like Rust's #[test] or pytest).

sourcefrog commented 1 week ago

Yes, I think it's an interesting idea. I think it's more closely connected to a coverage tool than to cargo-mutants, and it would probably require running each test separately like Nextest does.