search

gfredericks / test.chuck

A utility library for test.check
Eclipse Public License 1.0
215 stars 26 forks source link

Failing tests reported with :type :pass #32

Closed martinklepsch closed 8 years ago

martinklepsch commented 8 years ago

I think I'm seeing some sort of regression of #17 (works fine with 0.1.21 and then unexpected behavior w/ 0.1.22)

0.1.21 properly passes this to report functions:

{:type :fail, :file clojure_test.clj, :line 17, :expected (not-falsey-or-exception? (:result result)), :actual (not (not-falsey-or-exception? false)), :message {:result false, :seed 1447404832006, :failing-size 0, :num-tests 1, :fail [()], :shrunk {:total-nodes-visited 0, :depth 0, :result false, :smallest [()]}}}

whereas 0.1.22 passes a similar map but with :type :pass 

{:type :pass, :expected (not-exception? (:result result)), :actual (#object[com.gfredericks.test.chuck.clojure_test$not_exception_QMARK_ 0x15e4da62 com.gfredericks.test.chuck.clojure_test$not_exception_QMARK_@15e4da62] false), :message {:result false, :seed 1447404628802, :failing-size 0, :num-tests 1, :fail [()], :shrunk {:total-nodes-visited 0, :depth 0, :result false, :smallest [()]}}}

Stumbled upon this while working on collection-check so here's a reproduction case based on that: https://github.com/martinklepsch/collection-check-1/commits/zt-master

Edit: forgot to mention, this also happens with 0.2.0.

martinklepsch commented 8 years ago

Just discovered #29 which should probably fix this in one way or another.

martinklepsch commented 8 years ago

I guess we can close this one as it's solved by #29 in master. I created an alternative impl that is extensible for libraries like collection-check in #35