clojure.test integration

[cemerick]

Includes a defspec form for easily manifesting a property+args+default trial count as a clojure.test test var, and a first cut at trial and shrinking reporting using the clojure.test report multimethod.

Default reporting (i.e. silence) remains.

This fixes gh-2 and gh-4.

[reiddraper]

I'm having trouble getting the dot printing to work. In one case, I get a NPE, and in another, I just don't see any dots print. Here's the test I'm running:

(binding [simple-check.clojure-test/*report-trials* true]
  (defspec failing-spec 10000 (constantly true) (gen/int 100)))

that by itself never prints dots, and if I change the 1000 to 100 in simple-check.clojure-test/trial-report-dots I start seeing a NPE. Thoughts?

[cemerick]

The configuration vars only affect run(test)time; defspec doesn't capture those values. If you're going to use binding:

(defspec passing-spec 10000 (constantly true) (gen/int 100))

(binding [simple-check.clojure-test/*report-trials* true]
  (test-var #'passing-spec))

I think it'll probably be more common to alter-var-root those vars depending on the context.

I couldn't repro the NPE; do you have a stack trace?

[reiddraper]

The configuration vars only affect run(test)time; defspec doesn't capture those values.

Derp, makes sense.

I couldn't repro the NPE; do you have a stack trace?

Yes, Gist here.

[reiddraper]

Oh, and from running it within lein test

[cemerick]

That's a bizarre error; but, the only way I've found I can reproduce it is if I make simple-check.clojure-test-test/failing-spec passing…which, coincidentally, the snippet you pasted previously does. Any chance it's still floating around in your working copy?

[cemerick]

FYI, looks like the NPE is a leiningen bug involving test-ns-hook: technomancy/leiningen#1201

[reiddraper]

Any chance it's still floating around in your working copy?

Yep, just pushed this branch: repro/trial-report-NPE.

[cemerick]

This change makes the test fail by causing more dots to be emitted than are expected here. If you revert the former, all tests pass.

[cemerick]

That said, feel free to dictate the number-of-trials-per-dot. :-)

[reiddraper]

Aha, so the test fails because I didn't update the expected string, and then in turn, test-ns-hook failing causes the NPE in leiningen?

[cemerick]

Exactly. :-/

[reiddraper]

OK cool, I'll play with this some more soon but I think I'm gonna be +1 shortly.

[cemerick]

Sweet, thanks! :-D