search

clojure-expectations / lein-autoexpect

Run expecations when source or test files change
103 stars 12 forks source link

Keep failed tests as part of every test run until they are fixed #14

Closed noidi closed 10 years ago

noidi commented 10 years ago

Currently a green bar doesn't tell you that all tests pass. It tells you that the tests affected by the last change pass. To make sure there are no failing tests, you have to restart lein-autoexpect.

With this change lein-autoexpect reloads namespaces with failed tests to make them part of the next run, and a green bar means that all tests pass.

jakemcc commented 10 years ago

Thanks for noticing this. Definitely not the expected behavior.

I ended up poking around a bit in expectations because this behavior confused me. Because of how expect works when an expectation is run some metadata (:expectations/run true) is added to the var. This is then used to not rerun that expectation again without the expect sexp being reevaluated to create a new var.

My guess is the above is done to not run the same expectation multiple times in the same VM without revaluating the namespace (this behavior would facilitate running in a repl but not running the same expectation multiple times).

Clearly this isn't ideal behavior for something like lein-autoexpect. I took a different approach than your solution and end up clearing out the added metadata. Achieves a similar goal of this pull request but also causes all of the expectations to run which is the behavior that was intended when lein autoexpect runs the tests.

New functionality is in version 1.3.0.

Once again, thanks for the pull request. This is probably one of the top 3 pull requests I've ever received in that it does what it says, didn't break other functionality, and updated the change log.