Allow module-path configuration to contain multiple paths

[ThunderKey]

I had to estimate how many mutants survive in a huge project where around 400'000 mutations were found. For this I've randomly selected tests and only wanted to mutate the targets of those tests. With the excluded-modules this would have been a lot of work. For that I've adapted the code to allow module-path to be a list of paths, which will be used as mutation targets.

Since I've already done the work I thought you might be interested in it. If not feel free to discard this PR :)

Additionally, I would expect the tool to abort if the module-path is not found and not just do nothing. For an example see: https://github.com/sixty-north/cosmic-ray/blob/e5e0e3b4626da101975fa9ce2be3ce50966c97d2/tests/unittests/test_find_modules.py#L34-L36

Since this changes the behaviour I didn't want to just include it, but if you agree I can also add a commit that checks that the file/folder exists.

[abingham]

This is great, thanks!

Additionally, I would expect the tool to abort if the module-path is not found and not just do nothing.

This is probably better behavior than we have now. So if you've got a patch that does this, I'd be happy to consider it.

I had to estimate how many mutants survive in a huge project where around 400'000 mutations were found. For this I've randomly selected tests and only wanted to mutate the targets of those tests.

This is an interesting topic. Another way to approach it might be to write a filter that skips all but a random sampling of mutations, letting the user specify, say, a percentage of tests to run or something.

[ThunderKey]

This is probably better behavior than we have now. So if you've got a patch that does this, I'd be happy to consider it.

I've created #534 with a patch to verify that the paths exist.

This is an interesting topic. Another way to approach it might be to write a filter that skips all but a random sampling of mutations, letting the user specify, say, a percentage of tests to run or something.

That would also be an interesting approach. I guess the difficulty would be to detect which mutations belong to which test, without skipping too much that is not covered. Otherwise the resulting score will not be a reliable estimate.

[abingham]

I've created https://github.com/sixty-north/cosmic-ray/pull/534 with a patch to verify that the paths exist.

Thanks!

I guess the difficulty would be to detect which mutations belong to which test, without skipping too much that is not covered. Otherwise the resulting score will not be a reliable estimate.

Ah, I hadn't read your initial problem description closely enough. I don't tend to think of mutations as belonging to a particular test since, in general, mutations can change program behavior drastically enough that all sorts of "unrelated" tests might be affected. With that said, your approach of using the relationship between modules and their tests as a proxy is a practical one most of the time.

I think you could design the filter to a) skip all mutations in modules you're not interested in at all and then b) randomly skip those in modules of interest until you've driven the number of mutations to the target.

The only reason I belabor this idea at all is that I've learned to be reluctant to make changes to the "core" of CR when the work can be done with a filter program. I'm talking myself through my anxiety about making the core any more complex than it needs to be 😅

[ThunderKey]

That's a very nice idea with the filters. I'll have to read up more about them :)

This totally makes sense to keep the core as simple as it can be.