search

coala / projects

A website that showcases interesting projects, using Angular JS.
https://projects.coala.io/
GNU Affero General Public License v3.0
115 stars 221 forks source link

Introduce mutation testing #726

Open li-boxuan opened 5 years ago

li-boxuan commented 5 years ago

We don't have mutation testing used in any coala repository.

As http://pitest.org/ points out,

Traditional test coverage (i.e line, statement, branch, etc.) measures only which code is executed by your tests. It does not check that your tests are actually able to detect faults in the executed code. It is therefore only able to identify code that is definitely not tested.

The most extreme examples of the problem are tests with no assertions. Fortunately these are uncommon in most code bases. Much more common is code that is only partially tested by its suite. A suite that only partially tests code can still execute all its branches (examples).

As it is actually able to detect whether each statement is meaningfully tested, mutation testing is the gold standard against which all other types of coverage are measured.

As far as I am concerned, checkstyle community uses mutation testing, which seems to be quite powerful.

See available mutation testing tools for python: https://github.com/theofidry/awesome-mutation-testing Unfortunately, none of them is very popular, and I have no idea how well they work.

realJema commented 5 years ago

I will like to work on this project. are you the mentor @li-boxuan ?

li-boxuan commented 5 years ago

@realJema No, at least not for this year. And I am not sure whether this can be a suitable gsoc project.

KVGarg commented 5 years ago

In context to Implement metrics for coala, can this be added as a software metric for coala?