jimivdw / grunt-mutation-testing

JavaScript Mutation Testing as grunt plugin. Tests your tests by mutating the code.
MIT License
51 stars 11 forks source link

Mutation Testing Seems Broken with Karma + Dojo #60

Closed divido closed 8 years ago

divido commented 8 years ago

I'm looking into using mutation testing on a dojo-heavy project, so I started using grunt-mutation-testing. I'm using karma and jasmine, and have used karma's dojo framework plugin for support.

When I run the mutation task in grunt, it complains about tests failing prior to mutation.

I believe I've traced this down to an interaction between the KarmaServerManager and Karma's Server module, during the handoff of karma's config structure. One of the fields that grunt-mutation-testing overwrites is the "files", which it supplies as an array of strings. This is valid for Karma, but karma also supports arrays of objects of the form { pattern: "", included: false }. For interactions with dojo, it is important to have all dojo modules specified, but not included, and have a single "bootstrap" or similar function that is included.

Just prior to sending the config to Karma, I replaced it with a hard-coded value unique to my test project, and it worked correctly. I'm looking into a more general solution now, but I wanted to bring it to your attention in case I missed something or you had some suggestions on how to implement the fix.