search

jimivdw / grunt-mutation-testing

JavaScript Mutation Testing as grunt plugin. Tests your tests by mutating the code.
MIT License
51 stars 11 forks source link

nodeunit support #30

Open kupusc opened 9 years ago

kupusc commented 9 years ago

Hi,

Do you plan to add a support for the nodeunit?

Cheers, Adam

paysdoc commented 9 years ago

Hi Adam, Jimi and I talked about this. Neither of us has much experience with NodeUnit (yet). That is not to say that we won't support it - in fact we'd love to provide support for as many testing frameworks as possible. However, we're in the process of refactoring the project which includes modularizing it and, in doing so, making it possible to add all sorts of testing frameworks (and task runners like Gulp) more easily. That said, you're welcome to provide a pull request, but bear in mind that that would probably lead to rework once the refactoring is done. I hope to have the refactoring done by early September.

WFgLzxe7 commented 9 years ago

Ok, I started to implement this, and I have the following problem (note that I'm a total js newbie):

The mutated code throws sometimes a Fatal Error (it's quite obvious e.g. when accessing a property on an undefined object, which is undefined due to a particular mutation, etc.), and then the whole app just stops, which is not good. Did you come across the same problem with karma/mocha?

For now, I'm dealing with the problem by running the tests in child processes, which protects the main app from fatal errors (I'm able to recognize success/failure by reading the return code), but I have an impression that it slows down the execution quite a lot...

jimivdw commented 9 years ago

Hi Adam,

I personally do not know NodeUnit, but if that throws a fatal error upon encountering invalid code, I'm afraid there is no way around using subprocesses. This is at least what we have done for Karma.

If you encounter a huge performance loss due to the subprocesses, you might be doing something wrong in spawning or handling them. Theoretically, you shouldn't really be able to notice any performance loss when using subprocesses, but then again I don't know NodeUnit, and it might be that that simply doesn't handle the subprocesses well.

If I find the time, I'll have a look at what you've done so far and see if I can be of any help. Cannot make any promises though, busy times...

-Jimi

WFgLzxe7 commented 9 years ago

On 07/10/2015 10:19 PM, ext jimivdw wrote:

Hi Adam,

I personally do not know NodeUnit, but if that throws a fatal error upon encountering invalid code, I'm afraid there is no way around using subprocesses. This is at least what we have done for Karma.

If you encounter a huge performance loss due to the subprocesses, you might be doing something wrong in spawning or handling them. Theoretically, you shouldn't really be able to notice any performance loss when using subprocesses, but then again I don't know NodeUnit, and it might be that that simply doesn't handle the subprocesses well.

If I find the time, I'll have a look at what you've done so far and see if I can be of any help. Cannot make any promises though, busy times...

-Jimi

— Reply to this email directly or view it on GitHub https://github.com/jimivdw/grunt-mutation-testing/issues/30#issuecomment-120512047.

I'll continue with the subprocesses then - the performance loss was indeed caused by tests setups (some not mocked I/O operations) at the beginning of each process (so, running them in one process was much faster...).

Additionally, I implemented an option: "mutationTest:options:symlinks" for required folders that are quite big (e.g. node_modules), and copying them seems to be not really beneficial. Comma separated paths to folders are recreated as symbolic links to their original locations.

Thanks, Adam.