search

ansble / monument

event based http server for nodejs
http://monument.ansble.com
MIT License
34 stars 45 forks source link

Look at implementing generative tests for some parts of the API #203

Open designfrontier opened 8 years ago

designfrontier commented 8 years ago

https://github.com/graue/gentest

and the discussion here https://news.ycombinator.com/item?id=10070027

These aren't unit tests, they're generative tests. They're used for different things. Unit tests make assertions about how a program responds to a specific input. Generative tests make assertions about invariants in a program over a wide range of inputs. The workflow for using them is very different. A unit test suite contains a finite number of assertions and, assuming no bugs, should run in a relatively small (or at least bounded) amount of time. A generative test, however, usually can run forever by design. A typical workflow is to start a run overnight and see if it caught anything in the morning. If the generative suite finds any bugs, you turn the specific cases that caused the failures into unit tests and commit them.

They could be an interesting way of hardening the system as a whole.

designfrontier commented 8 years ago

It would be nice to open some specific issues for areas where this sort of testing makes sense. Hence the first-contribution flag. Bonus points if those tickets come with PRs to implement that generative tests.

designfrontier commented 8 years ago

Another library: https://github.com/mcandre/node-quickcheck

designfrontier commented 8 years ago

And a third https://github.com/jsverify/jsverify this one looks the most complete at the moment