gfredericks / test.chuck

A utility library for test.check
Eclipse Public License 1.0
214 stars 26 forks source link

Unable to use an empty binding vector in v0.2.2-SNAPSHOT #40

Closed sandhu closed 8 years ago

sandhu commented 8 years ago

I've been using v0.2.0 and was able to integrate specific unit test cases into the generative tests by doing something like

(deftest foo
  (checking "checking for nil" 1
            []
            (is (nil? (identity nil)))))

In the master branch I am unable to do that and get the following error on compile.

  Show: Clojure Java REPL Tooling Duplicates All  (96 frames hidden)

2. Unhandled clojure.lang.Compiler$CompilerException
   Error compiling:
   fn_test.cljc:55:3 Unsupported binding form:

                 Compiler.java: 6730  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5296  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3925  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6721  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3195  clojure.lang.Compiler$VectorExpr/parse
                 Compiler.java: 6526  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 6179  clojure.lang.Compiler$LetExpr$Parser/parse
                 Compiler.java: 6723  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5296  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3925  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6721  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5296  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3925  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6721  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3050  clojure.lang.Compiler$MapExpr/parse
                 Compiler.java: 6532  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java:  576  clojure.lang.Compiler$DefExpr$Parser/parse
                 Compiler.java: 6723  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 6786  clojure.lang.Compiler/eval
                 Compiler.java: 6745  clojure.lang.Compiler/eval
                      core.clj: 3081  clojure.core/eval
                      main.clj:  240  clojure.main/repl/read-eval-print/fn
                      main.clj:  240  clojure.main/repl/read-eval-print
                      main.clj:  258  clojure.main/repl/fn
                      main.clj:  258  clojure.main/repl
                   RestFn.java: 1523  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   87  clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn
                      AFn.java:  152  clojure.lang.AFn/applyToHelper
                      AFn.java:  144  clojure.lang.AFn/applyTo
                      core.clj:  630  clojure.core/apply
                      core.clj: 1868  clojure.core/with-bindings*
                   RestFn.java:  425  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   85  clojure.tools.nrepl.middleware.interruptible-eval/evaluate
        interruptible_eval.clj:  222  clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
        interruptible_eval.clj:  190  clojure.tools.nrepl.middleware.interruptible-eval/run-next/fn
                      AFn.java:   22  clojure.lang.AFn/run
       ThreadPoolExecutor.java: 1142  java.util.concurrent.ThreadPoolExecutor/runWorker
       ThreadPoolExecutor.java:  617  java.util.concurrent.ThreadPoolExecutor$Worker/run
                   Thread.java:  745  java.lang.Thread/run

1. Caused by java.lang.Exception
   Unsupported binding form:

                      core.clj: 4293  clojure.core/destructure/pb
                      core.clj: 4294  clojure.core/destructure/process-entry
                      core.clj:  909  clojure.core/reduce1
                      core.clj: 4299  clojure.core/destructure
                      core.clj: 4312  clojure.core/let
                   RestFn.java:  467  clojure.lang.RestFn/invoke
                      Var.java:  394  clojure.lang.Var/invoke
                      AFn.java:  165  clojure.lang.AFn/applyToHelper
                      Var.java:  700  clojure.lang.Var/applyTo
                 Compiler.java: 6631  clojure.lang.Compiler/macroexpand1
                 Compiler.java: 6709  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5296  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3925  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6721  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3195  clojure.lang.Compiler$VectorExpr/parse
                 Compiler.java: 6526  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 6179  clojure.lang.Compiler$LetExpr$Parser/parse
                 Compiler.java: 6723  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5296  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3925  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6721  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3791  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6725  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 5861  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5296  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3925  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6721  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 3050  clojure.lang.Compiler$MapExpr/parse
                 Compiler.java: 6532  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java:  576  clojure.lang.Compiler$DefExpr$Parser/parse
                 Compiler.java: 6723  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6524  clojure.lang.Compiler/analyze
                 Compiler.java: 6485  clojure.lang.Compiler/analyze
                 Compiler.java: 6786  clojure.lang.Compiler/eval
                 Compiler.java: 6745  clojure.lang.Compiler/eval
                      core.clj: 3081  clojure.core/eval
                      main.clj:  240  clojure.main/repl/read-eval-print/fn
                      main.clj:  240  clojure.main/repl/read-eval-print
                      main.clj:  258  clojure.main/repl/fn
                      main.clj:  258  clojure.main/repl
                   RestFn.java: 1523  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   87  clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn
                      AFn.java:  152  clojure.lang.AFn/applyToHelper
                      AFn.java:  144  clojure.lang.AFn/applyTo
                      core.clj:  630  clojure.core/apply
                      core.clj: 1868  clojure.core/with-bindings*
                   RestFn.java:  425  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   85  clojure.tools.nrepl.middleware.interruptible-eval/evaluate
        interruptible_eval.clj:  222  clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
        interruptible_eval.clj:  190  clojure.tools.nrepl.middleware.interruptible-eval/run-next/fn
                      AFn.java:   22  clojure.lang.AFn/run
       ThreadPoolExecutor.java: 1142  java.util.concurrent.ThreadPoolExecutor/runWorker
       ThreadPoolExecutor.java:  617  java.util.concurrent.ThreadPoolExecutor$Worker/run
                   Thread.java:  745  java.lang.Thread/run
gfredericks commented 8 years ago

I'm confused about the use case here -- why wouldn't your example just be written without test.check at all?

(deftest foo
  (testing "checking for nil"
    (is (nil? (identity nil)))))
sandhu commented 8 years ago

I could certainly do it the way you've shown and the consistency argument wouldn't be strong enough on it's own.

I'd structured my tests the way I've described above (using checking) and it did work until I tried to upgrade.

At this point the strongest case I could make would be for backwards compatibility. Fixing this regression would save me from having to go through my code base and "fix" the tests written this way.

sandhu commented 8 years ago

It's worth pointing out that this is a great library and has worked flawlessly up until this point for me.

gfredericks commented 8 years ago

I'm not opposed to a change that supports this. I think the difference was caused by switching from test.check's prop/for-all to test.chuck's prop/for-all as the underlying expansion, so the root cause is that chuck.prop/for-all doesn't support empty bindings. At a glance it seems not trivial to fix this, but it might be easy after five minutes of thinking or so.

I can get to this before too long, if nobody else is interested in trying. Would probably be a fun exercise for anybody looking for that sort of thing.

sandhu commented 8 years ago

Took some time to dig into it this morning and the issue appears to be with gen'/for where it is using destructing to take apart the bindings

(let [[k1 v1 & [k2 v2 & even-more :as more]] bindings] ...)

One quick "fix" is to guard against that as follows:

(defmacro for-all
  "Alternative version of clojure.test.check.properties/for-all where
  the binding forms are interpreted as per
  com.gfredericks.test.chuck.generators/for."
  [bindings expr]
  `(prop/for-all ~(if (empty? bindings)
                    `[{:syms []} []]
                    (let [bound-names (for-bindings bindings)
                          quoted-names (map #(list 'quote %) bound-names)]
                      `[{:syms [~@bound-names]}
                        (gen'/for ~bindings
                                  (with-meta
                                    ~(zipmap quoted-names bound-names)
                                    {::for-all-bindings-map true}))]))
                 ~expr))

The proper solution would be to change for so that it does not use destructing or add a nil check.

sandhu commented 8 years ago

Thank you. Very much appreciated.

gfredericks commented 8 years ago

Released in 0.2.3.