Closed abingham closed 2 years ago
For that exec
would need to start executing the tests in random order again.
For that exec would need to start executing the tests in random order again.
I don't understand this. If I have a fully-executed session, the order or mutation seems irrelevant for calculating survival rate. For many cases, this final survival rate is what's important, and it's what cr-rate was initially created for.
Because when you're using it to actually estimate the survival rate from a partial run, you need to to have unbiased sample: selected at random from the population of all mutations. If the sample is just from one module, that may seriously under- or over-estimate the survival rate of the whole application, as that particular module can have significantly worse or better test coverage than other modules. If you just want the survival rate, you can simply look at last line of cr-report anyway.
Side note: if you run cr-rate and specify the --confidence 99.0
(or other valid), then it works just fine, the problem is the default value for the option parser
the problem is the default value for the option parser
Ah, right you are! This was much ado about nothing. I've updated the default argument value and fixed the tests in the process. Thanks for pointing that out.
cr-rate doesn't currently do its most basic task: report the survival rate. It has all sorts of stuff about estimates and confidence intervals injected into it, and - while I'm sure these are useful - they shouldn't be required by default.