Prevent module overwrite, allow clear/remove based on configuration

[lbragile]

Is your feature request related to a problem? Please describe. My current setup has ~1600 mutants across 9 files. To reduce mutation testing time (takes 12 hours total), I parallelize these in my GitHub workflow (slowest takes 6 hours). Some of these files are in the same folder eg. "App/App_functions.js" and "App/App_helpers.js". During mutation testing the natural module to make for both (in dashboard) is "App", but since they end at different times, one overwrites the other.

Describe the solution you'd like Specifying a module for dashboard reporter should not overwrite existing files in that module. Maybe include an additional config parameter like "overwriteModule: Boolean" so that the user can decide whether or not that is what they want.

Describe alternatives you've considered I created different modules for each file, in total I had 9 unique modules for my 9 files - this is not ideal.

Additional context When creating new modules, the old modules remain in the dashboard and there seems to be no way to clear/remove them so they end up interfering with the mutation score. It would be great if a feature for this was provided as well. Maybe add another config variable in the dashboard reporter that removes specific module names. Eg. "existingModules.names: string[]" // contains module names to consider "existingModules.actions: {'overwrite', 'clear', 'remove'}" // what to do with these modules

[nicojs]

I've transferred this issue to https://github.com/stryker-mutator/stryker-dashboard, since this feature should first be build into the dashboard before mutation testing frameworks could support it..

Maybe include an additional config parameter like "overwriteModule: Boolean" so that the user can decide whether or not that is what they want.

Maybe we can use the HTTP patch method here. It would merge the given result into the existing one.

I personally don't have time to work on this in the short term. You can either create a PR yourself, keep using your current workaround, or create your own "patch" workaround by combining results on the client (fetching the current JSON, merging JSON's client-side, and uploading the result with a POST).

[lbragile]

@nicojs thank you for the quick (and honest) response.

I will experiment with a few ideas I have such as mutating an empty file and using it to overwrite an existing module with the aim to at least remove contamination from old modules. Also, I doubt this will work but I will attempt to disconnect and then reconnect my repository to the dashboard.

When I have some free time, I will look into attempting to contribute via a PR, but I will be honest in saying that this will be a big challenge for me since I’ve not used TS yet and this project is massive.

Also, I will do my best to document what works for me as well as provide any necessary scripts here if that’s acceptable.

Lastly, I want to thank you and everyone involved with this project as using mutation testing has really developed me as a “programmer” and completely changed the way I test my code. Your efforts are greatly appreciated!

[lbragile]

Update 1:

I figured out a "hack" to remove modules from the dashboard without any extra code. Just a single empty file is required. Information available here: https://github.com/stryker-mutator/stryker-dashboard/issues/173#issuecomment-790922577

Now just need to figure out how to not overwrite existing files in a module.

[nicojs]

Adding a HTTP DELETE functionality should be pretty straightforward and we actually also need that for stryker itself (a lot of old modules there as well 😉) so that will have slightly higher priority

[xandervedder]

If #246 is implemented, would this resolve this issue?

If yes, I would like to track it in #246 instead 😃.

[rouke-broersma]

If #246 is implemented, would this resolve this issue?

If yes, I would like to track it in #246 instead 😃.

The actual ask seems to be some kind of incremental mode or something that goes beyond just separate modules. I'm not really understanding how you end up with 1600 mutations in just 9 files. To me it more or less sounds like a lack of separations in the source project and I'm not sure it's in our interest to try and find a good fix for this outlier. I don't think this is a concern for the majority of users.