adolgert / UnitTestDesign.jl

Increases unit test coverage with fewer test cases using all-pairs and other covering arrays.
http://computingkitchen.com/UnitTestDesign.jl/stable/
MIT License
15 stars 2 forks source link

Doc/downsides #36

Closed adolgert closed 2 years ago

adolgert commented 3 years ago

These commits are for #33. In this you will find:

  1. "False sense of security..." - Put paragraph near end that says test case numbers go up but that you still need to do the software engineering methods like counting bug report rates. I get that people might get wowed by the number of tests. They mean something different from numbers of manual test.
  2. "Excessive tests that become hard to update as the code changes" - I disagree. Here, you get more coverage with less code, compared to manual testing. You write one test and run it inside a loop over test cases. That seems less brittle to me. Do you take this another way?
  3. "Avoiding need for a more robust contracts/interfaces/test-suites" - You'll see a change to the introduction, with a reference to a nice paper on software testability. Really, it's a cute paper. These techniques are 100% necessary, and I think about them a lot. What I'd like to do here is ask, once those steps are done, what can we work with from our testing framework?
codecov-commenter commented 3 years ago

Codecov Report

Merging #36 (0129cde) into main (767726f) will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main      #36   +/-   ##
=======================================
  Coverage   90.46%   90.46%           
=======================================
  Files           9        9           
  Lines         860      860           
=======================================
  Hits          778      778           
  Misses         82       82           

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 767726f...0129cde. Read the comment docs.

oxinabox commented 2 years ago

Thanks LGTM