search

adolgert / UnitTestDesign.jl

Increases unit test coverage with fewer test cases using all-pairs and other covering arrays.
http://computingkitchen.com/UnitTestDesign.jl/stable/
MIT License
15 stars 2 forks source link

mention Vimes.jl re:mutation testing #30

Closed oxinabox closed 2 years ago

adolgert commented 3 years ago

Man, I thought about linking to Vimes.jl. I worked a lot with it because its design is compact and skilled with code parsing and modification. I didn't link because Vimes.jl is set up for "mutation testing," and I was thinking about "mutation coverage." Mutation testing tells you how many and which mutants weren't killed. It doesn't have strong support in the literature because a large percentage of mutants cause false positives or false negatives. Mutation coverage simplifies the problem by counting only those mutants that ran and were found by at least one test. Then it asks which subset of your tests would find all of that subset of mutants. It's used for test selection, where you reduce the number of tests.

I need to edit this section, one way or another. I'll see if I can add to this PR. I love that you made PRs.

adolgert commented 3 years ago

I updated the PR and have one main question: Is this "test selection" section something I should delete because it isn't helpful? I added it because there is a workflow where you generate tests with combinatorial testing and then select a subset.

codecov-commenter commented 3 years ago

Codecov Report

Merging #30 (b928b36) into main (2c51b85) will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main      #30   +/-   ##
=======================================
  Coverage   90.46%   90.46%           
=======================================
  Files           9        9           
  Lines         860      860           
=======================================
  Hits          778      778           
  Misses         82       82

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2c51b85...b928b36. Read the comment docs.