search

boxed / mutmut

Mutation testing system
https://mutmut.readthedocs.io
BSD 3-Clause "New" or "Revised" License
932 stars 112 forks source link

[Feature Request] Random Sampling of Mutations #306

Closed johnaugustflynn closed 1 month ago

johnaugustflynn commented 9 months ago

Given how expensive mutation testing is, I think it would be valuable to have the option of only testing a random sample of mutations. This could still give you a decent idea of your test quality and some common themes of surviving mutants, without the lengthy runtime required for exhaustively testing thousands of mutants. Some other mutation testing packages already support this, but I like mutmut and would love to see it here.

I am imagining something like mutmut run --random-sample 0.4 --random-seed 42

I'd be open to contributing a PR if maintainer(s) are interested.

boxed commented 9 months ago

Another way to think about this would be shuffle. Since mutants are a fixed set, sampling is the same as shuffling before starting really.

johnaugustflynn commented 9 months ago

Makes sense. So does that mean you are open to this?

boxed commented 9 months ago

Yes. For sure.

boxed commented 1 month ago

Duplicate of #278

boxed commented 1 month ago

Also, I released mutmut 3 which WAY faster, so this issue isn't as critical anymore I think.