search

marick / Midje

Midje provides a migration path from clojure.test to a more flexible, readable, abstract, and gracious style of testing
MIT License
1.69k stars 129 forks source link

`tabular` + `fact` interaction can create invalid var names #472

Open vemv opened 3 years ago

vemv commented 3 years ago

Hi! I found the following issue.

We can observe that Midje emits delicate-looking var names:

matcher-combinators.core-test> (tabular (fact "Foo") ?matcher prefix embeds)
;; the whole thing below is a var object!
#'matcher-combinators.core-test/#:midje{:source (quote (tabular (fact "Foo") ?matcher prefix embeds)), :guid "5ff8c9530e9cdb7f77c312813c66f4aca8cbb9ab"}354442

Which becomes problematic when running more substantial code:

matcher-combinators.core-test> (tabular
                                (fact "Providing seq/map matcher with incorrect input leads to automatic mismatch"
                                      (core/match (?matcher 1) 1)
                                      => (just {::result/type   :mismatch
                                                ::result/value  (sweet/contains {:expected-type-msg
                                                                                 #(str/starts-with? % (-> ?matcher var meta :name str))
                                                                                 :provided
                                                                                 "provided: 1"})
                                                ::result/weight number?}))
                                ?matcher
                                prefix
                                embeds)
Syntax error macroexpanding clojure.core/defn at (*cider-repl 127.0.0.1*:1:32).
#:midje{:source (quote (tabular (fact "Providing seq/map matcher with incorrect input leads to automatic mismatch" (core/match (?matcher 1) 1) => (just #:matcher-combinators.result{:type :mismatch, :value (sweet/contains {:expected-type-msg (fn* [p1__354461#] (str/starts-with? p1__354461# (-> ?matcher var meta :name str))), :provided "provided: 1"}), :weight number?})) ?matcher prefix embeds)), :guid "171e8922a091d677c0175735d17f1a1dffb6df1a"}354464 - failed: simple-symbol? at: [:fn-name] spec: :clojure.core.specs.alpha/defn-args

Could you please look into emitting more var names having a less risky character set?

Thanks - V

philomates commented 3 years ago

Hi @vemv, I ran the code you provided and didn't encounter any issues and I couldn't quite understand what is coming up from your description. Could you provide another example or maybe some more context?

I ran the following and it passed fine:

(ns matcher-combinators.scratch
  (:require [midje.sweet :refer :all]
            [clojure.string :as str]
            [matcher-combinators.result :as result]
            [matcher-combinators.midje :refer [match]]
            [matcher-combinators.core :as core]
            [matcher-combinators.matchers :as matchers]))

(tabular
  (fact "Providing seq/map matcher with incorrect input leads to automatic mismatch"
        (core/match (?matcher 1) 1)
        => (just {::result/type   :mismatch
                  ::result/value  (contains {:expected-type-msg
                                             #(str/starts-with? % (-> ?matcher var meta :name str))
                                             :provided
                                             "provided: 1"})
                  ::result/weight number?}))
  ?matcher
  matchers/prefix
  matchers/embeds)

That said, I did get tripped up because tabular swallows underlying exceptions, so typos often get obfuscated, so:

(tabular
  (fact "Providing seq/map matcher with incorrect input leads to automatic mismatch"
        (core/match (?matcher 1) 1)
        => any?)
  ?matcher
  i-dont-exist/equals)

results in 

FAIL at (scratch.clj:23)
    Midje could not understand something you wrote:
        It looks like the table has headings, but no values:

Is that somehow related to what you are getting at?

vemv commented 3 years ago

Thanks for the response. Indeed the matcher-combinators project surely has a green build and therefore the issue cannot be always reproduced.

However, did you check out the names of the var objects that get created out of a tabular + fact? If they contain a variety of odd characters, whitespace etc then you might see how the issue is plausible.

philomates commented 3 years ago

well, a green build doesn't always mean a correct implementation :)

but I'm still not following. A code snippet that shows the odd characters/whitespace you are referring to would be helpful.

For example, I macroexpanded the tabular but didn't see anything fishy

(ns matcher-combinators.scratch
  (:require [midje.sweet :refer :all]
            [clojure.string :as str]
            [matcher-combinators.result :as result]
            [matcher-combinators.midje :refer [match]]
            [matcher-combinators.core :as core]
            [matcher-combinators.matchers :as matchers]))

(clojure.pprint/pprint
  (macroexpand
    `(tabular
       (fact "Providing seq/map matcher with incorrect input leads to automatic mismatch"
             (core/match (?matcher 1) 1)
             => (just {::result/type   :mismatch
                       ::result/value  (contains {:expected-type-msg
                                                  #(str/starts-with? % (-> ?matcher var meta :name str))
                                                  :provided
                                                  "provided: 1"})
                       ::result/weight number?}))
       ?matcher
       mzz/equals
       matchers/prefix
       matchers/embeds)))
;; prints a huge syntax form that looks standard enough to me
vemv commented 3 years ago

Hi again!

Did you check out the name of the var that gets created by evaluating (tabular (fact ... in a repl?

As mentioned, this var name seems fishy in itself, and probably the easiest thing that both of us can reproduce.

philomates commented 3 years ago

hmm, I'm not seeing any vars created when I inspect the namespace using ns-interns. And when I paste the example code in the repl I get the following:

user=> (tabular (fact "Foo") ?matcher matchers/prefix matchers/embeds)
true
user=> (def x (tabular (fact "Foo") ?matcher matchers/prefix matchers/embeds))
#'user/x
user=> x
true
vemv commented 3 years ago

Alright, I'll try to provide a repro repo, as this might be related to an unlucky combination of deps or such

genmeblog commented 3 years ago

I've just hit by this in tests for tablecloth (https://github.com/scicloj/tablecloth/blob/master/test/tablecloth/api/reshape_test.clj#L9)

When I run lein midje everything looks ok. But when I evaluate them from Cider/Emacs (C-c C-k) I'm getting the compiler exception.

Maybe something wrong with Cider?

;; Connected to nREPL server - nrepl://localhost:34043
;; CIDER 1.1.0snapshot (package: 20210213.1151), nREPL 0.8.3
;; Clojure 1.10.2, Java 14.0.2
[...]
;;  Startup: /home/XXX/bin/lein update-in :dependencies conj \[nrepl/nrepl\ \"0.8.3\"\] -- update-in :plugins conj \[refactor-nrepl\ \"2.5.1\"\] -- update-in :plugins conj \[com.billpiel/sayid\ \"0.1.0\"\] -- update-in :plugins conj \[cider/cider-nrepl\ \"0.25.9\"\] -- update-in :plugins conj \[nubank/midje-nrepl\ \"1.2.0-SNAPSHOT\"\] -- repl :headless :host localhost

It happens for all of tests in reshape_test file, also for the following:

(fact "asdf" (tabular (fact (seq ?n) => ?v)
                      ?n ?v
                      [1 2 3] [1 2 3]))
2. Unhandled clojure.lang.Compiler$CompilerException
   Error compiling test/tablecloth/api/reshape_test.clj at (9:1)
   #:clojure.error{:phase :macro-syntax-check,
                   :line 9,
                   :column 1,
                   :source
                   "/home/XXX/clojure/tablecloth/test/tablecloth/api/reshape_test.clj",
                   :symbol clojure.core/defn}
             Compiler.java: 6976  clojure.lang.Compiler/checkSpecs
             Compiler.java: 6992  clojure.lang.Compiler/macroexpand1
                  core.clj: 4012  clojure.core/macroexpand-1
                  core.clj: 4014  clojure.core/macroexpand
                  core.clj: 4022  clojure.core/macroexpand
                  core.clj: 4014  clojure.core/macroexpand
               tabular.clj:   71  midje.parsing.0-to-fact-form.tabular/parse/fn
        error_handling.clj:   38  midje.parsing.util.error-handling/parse-and-catch-failure
        error_handling.clj:   17  midje.parsing.util.error-handling/parse-and-catch-failure
               tabular.clj:   66  midje.parsing.0-to-fact-form.tabular/parse
               tabular.clj:   65  midje.parsing.0-to-fact-form.tabular/parse
                 sweet.clj:  221  midje.sweet/tabular
                 sweet.clj:  208  midje.sweet/tabular
               RestFn.java:  142  clojure.lang.RestFn/applyTo
                  Var.java:  705  clojure.lang.Var/applyTo
             Compiler.java: 6997  clojure.lang.Compiler/macroexpand1
                  core.clj: 4012  clojure.core/macroexpand-1
                  core.clj: 4014  clojure.core/macroexpand
                  core.clj: 4014  clojure.core/macroexpand
                 facts.clj:  121  midje.parsing.1-to-explicit-form.facts/midjcoexpand
                 facts.clj:  108  midje.parsing.1-to-explicit-form.facts/midjcoexpand
                  core.clj: 2759  clojure.core/map/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                   RT.java:  535  clojure.lang.RT/seq
                  core.clj:  139  clojure.core/seq
                  core.clj: 2750  clojure.core/map/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                 Cons.java:   39  clojure.lang.Cons/next
       PersistentList.java:   37  clojure.lang.PersistentList$Primordial/doInvoke
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj:  662  clojure.core/apply
              laziness.clj:   15  midje.util.laziness/eagerly
              laziness.clj:    5  midje.util.laziness/eagerly
                 facts.clj:  124  midje.parsing.1-to-explicit-form.facts/midjcoexpand
                 facts.clj:  108  midje.parsing.1-to-explicit-form.facts/midjcoexpand
                 facts.clj:  162  midje.parsing.1-to-explicit-form.facts/expand-fact-body
                 facts.clj:  154  midje.parsing.1-to-explicit-form.facts/expand-fact-body
                 facts.clj:  200  midje.parsing.1-to-explicit-form.facts/complete-fact-transformation
                 facts.clj:  196  midje.parsing.1-to-explicit-form.facts/complete-fact-transformation
                 sweet.clj:  194  midje.sweet/fact/fn
        error_handling.clj:   41  midje.parsing.util.error-handling/parse-and-catch-failure
        error_handling.clj:   17  midje.parsing.util.error-handling/parse-and-catch-failure
                 sweet.clj:  186  midje.sweet/fact
                 sweet.clj:  170  midje.sweet/fact
               RestFn.java:  142  clojure.lang.RestFn/applyTo
                  Var.java:  705  clojure.lang.Var/applyTo
             Compiler.java: 6997  clojure.lang.Compiler/macroexpand1
             Compiler.java: 7079  clojure.lang.Compiler/macroexpand
             Compiler.java: 7165  clojure.lang.Compiler/eval
             Compiler.java: 7640  clojure.lang.Compiler/load
                      REPL:    1  user/eval46941
                      REPL:    1  user/eval46941
             Compiler.java: 7181  clojure.lang.Compiler/eval
             Compiler.java: 7136  clojure.lang.Compiler/eval
                  core.clj: 3202  clojure.core/eval
                  core.clj: 3198  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1977  clojure.core/with-bindings*
                  core.clj: 1977  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj:  662  clojure.core/apply
                regrow.clj:   20  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  832  java.lang.Thread/run
vemv commented 3 years ago

But when I evaluate them from Cider/Emacs (C-c C-k) I'm getting compiler exception.

We both use CIDER so yeah that's a lead :)

Could you post the root exception? I think you can obtain it via (.getCause *e)

genmeblog commented 3 years ago

Sure:

Syntax error macroexpanding clojure.core/defn at (test/tablecloth/api/reshape_test.clj:9:1).
(api/column-names ds)45000 - failed: simple-symbol? at: [:fn-name] spec: :clojure.core.specs.alpha/defn-args
user> (.getCause *e)
#error {
 :cause "Call to clojure.core/defn did not conform to spec."
 :data #:clojure.spec.alpha{:problems [{:path [:fn-name], :pred clojure.core/simple-symbol?, :val (api/column-names ds)45000, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/defn-args], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2510 0x4e253f65 "clojure.spec.alpha$regex_spec_impl$reify__2510@4e253f65"], :value ((api/column-names ds)45000 [] (midje.checking.facts/creation-time-check (clojure.core/with-meta (clojure.core/fn [] (midje.parsing.util.wrapping/midje-wrapped (midje.data.prerequisite-state/with-installed-fakes (midje.parsing.1-to-explicit-form.parse-background/background-fakes) (midje.parsing.util.wrapping/midje-wrapped (midje.checking.checkables/check-one (clojure.core/merge {:description (midje.data.nested-facts/descriptions), :expected-result-form (quote (quote (:$column :$value))), :check-expectation :expect-match, :midje.parsing.lexical-maps/a-midje-checkable-map? true, :function-under-test (clojure.core/fn [] (api/column-names ds)), :expected-result (quote (:$column :$value)), :position (pointer.core/line-number-known 13), :namespace clojure.core/*ns*} {:arrow (quote =>), :call-form (quote (api/column-names ds))} (clojure.core/hash-map :position (pointer.core/line-number-known 13) :position (pointer.core/line-number-known 13))) []))))) (clojure.core/merge #:midje{:guid "e09930609b3757665a0418142994d1824e439d4e", :source (quote (fact (api/column-names ds) => (quote (:$column :$value)) :position (pointer.core/line-number-known 13))), :namespace (quote tablecloth.api.reshape-test), :file "/home/XXX/clojure/tablecloth/test/tablecloth/api/reshape_test.clj", :line 13} #:midje{:top-level-fact? false})))), :args ((api/column-names ds)45000 [] (midje.checking.facts/creation-time-check (clojure.core/with-meta (clojure.core/fn [] (midje.parsing.util.wrapping/midje-wrapped (midje.data.prerequisite-state/with-installed-fakes (midje.parsing.1-to-explicit-form.parse-background/background-fakes) (midje.parsing.util.wrapping/midje-wrapped (midje.checking.checkables/check-one (clojure.core/merge {:description (midje.data.nested-facts/descriptions), :expected-result-form (quote (quote (:$column :$value))), :check-expectation :expect-match, :midje.parsing.lexical-maps/a-midje-checkable-map? true, :function-under-test (clojure.core/fn [] (api/column-names ds)), :expected-result (quote (:$column :$value)), :position (pointer.core/line-number-known 13), :namespace clojure.core/*ns*} {:arrow (quote =>), :call-form (quote (api/column-names ds))} (clojure.core/hash-map :position (pointer.core/line-number-known 13) :position (pointer.core/line-number-known 13))) []))))) (clojure.core/merge #:midje{:guid "e09930609b3757665a0418142994d1824e439d4e", :source (quote (fact (api/column-names ds) => (quote (:$column :$value)) :position (pointer.core/line-number-known 13))), :namespace (quote tablecloth.api.reshape-test), :file "/home/XXX/clojure/tablecloth/test/tablecloth/api/reshape_test.clj", :line 13} #:midje{:top-level-fact? false}))))}
 :via
 [{:type clojure.lang.ExceptionInfo
   :message "Call to clojure.core/defn did not conform to spec."
   :data #:clojure.spec.alpha{:problems [{:path [:fn-name], :pred clojure.core/simple-symbol?, :val (api/column-names ds)45000, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/defn-args], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2510 0x4e253f65 "clojure.spec.alpha$regex_spec_impl$reify__2510@4e253f65"], :value ((api/column-names ds)45000 [] (midje.checking.facts/creation-time-check (clojure.core/with-meta (clojure.core/fn [] (midje.parsing.util.wrapping/midje-wrapped (midje.data.prerequisite-state/with-installed-fakes (midje.parsing.1-to-explicit-form.parse-background/background-fakes) (midje.parsing.util.wrapping/midje-wrapped (midje.checking.checkables/check-one (clojure.core/merge {:description (midje.data.nested-facts/descriptions), :expected-result-form (quote (quote (:$column :$value))), :check-expectation :expect-match, :midje.parsing.lexical-maps/a-midje-checkable-map? true, :function-under-test (clojure.core/fn [] (api/column-names ds)), :expected-result (quote (:$column :$value)), :position (pointer.core/line-number-known 13), :namespace clojure.core/*ns*} {:arrow (quote =>), :call-form (quote (api/column-names ds))} (clojure.core/hash-map :position (pointer.core/line-number-known 13) :position (pointer.core/line-number-known 13))) []))))) (clojure.core/merge #:midje{:guid "e09930609b3757665a0418142994d1824e439d4e", :source (quote (fact (api/column-names ds) => (quote (:$column :$value)) :position (pointer.core/line-number-known 13))), :namespace (quote tablecloth.api.reshape-test), :file "/home/XXX/clojure/tablecloth/test/tablecloth/api/reshape_test.clj", :line 13} #:midje{:top-level-fact? false})))), :args ((api/column-names ds)45000 [] (midje.checking.facts/creation-time-check (clojure.core/with-meta (clojure.core/fn [] (midje.parsing.util.wrapping/midje-wrapped (midje.data.prerequisite-state/with-installed-fakes (midje.parsing.1-to-explicit-form.parse-background/background-fakes) (midje.parsing.util.wrapping/midje-wrapped (midje.checking.checkables/check-one (clojure.core/merge {:description (midje.data.nested-facts/descriptions), :expected-result-form (quote (quote (:$column :$value))), :check-expectation :expect-match, :midje.parsing.lexical-maps/a-midje-checkable-map? true, :function-under-test (clojure.core/fn [] (api/column-names ds)), :expected-result (quote (:$column :$value)), :position (pointer.core/line-number-known 13), :namespace clojure.core/*ns*} {:arrow (quote =>), :call-form (quote (api/column-names ds))} (clojure.core/hash-map :position (pointer.core/line-number-known 13) :position (pointer.core/line-number-known 13))) []))))) (clojure.core/merge #:midje{:guid "e09930609b3757665a0418142994d1824e439d4e", :source (quote (fact (api/column-names ds) => (quote (:$column :$value)) :position (pointer.core/line-number-known 13))), :namespace (quote tablecloth.api.reshape-test), :file "/home/XXX/clojure/tablecloth/test/tablecloth/api/reshape_test.clj", :line 13} #:midje{:top-level-fact? false}))))}
   :at [clojure.spec.alpha$macroexpand_check invokeStatic "alpha.clj" 712]}]
 :trace
 [[clojure.spec.alpha$macroexpand_check invokeStatic "alpha.clj" 712]
  [clojure.spec.alpha$macroexpand_check invoke "alpha.clj" 704]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.lang.Var applyTo "Var.java" 705]
  [clojure.lang.Compiler checkSpecs "Compiler.java" 6974]
  [clojure.lang.Compiler macroexpand1 "Compiler.java" 6992]
  [clojure.core$macroexpand_1 invokeStatic "core.clj" 4012]
  [clojure.core$macroexpand invokeStatic "core.clj" 4014]
  [clojure.core$macroexpand invokeStatic "core.clj" 4022]
  [clojure.core$macroexpand invoke "core.clj" 4014]
  [midje.parsing.1_to_explicit_form.facts$midjcoexpand invokeStatic "facts.clj" 120]
  [midje.parsing.1_to_explicit_form.facts$midjcoexpand invoke "facts.clj" 108]
  [clojure.core$map$fn__5885 invoke "core.clj" 2757]
  [clojure.lang.LazySeq sval "LazySeq.java" 42]
  [clojure.lang.LazySeq seq "LazySeq.java" 51]
  [clojure.lang.RT seq "RT.java" 535]
  [clojure.core$seq__5420 invokeStatic "core.clj" 139]
  [clojure.core$map$fn__5885 invoke "core.clj" 2750]
  [clojure.lang.LazySeq sval "LazySeq.java" 42]
  [clojure.lang.LazySeq seq "LazySeq.java" 51]
  [clojure.lang.RT seq "RT.java" 535]
  [clojure.core$seq__5420 invokeStatic "core.clj" 139]
  [clojure.core$apply invokeStatic "core.clj" 662]
  [clojure.core$apply invoke "core.clj" 662]
  [midje.util.laziness$eagerly invokeStatic "laziness.clj" 15]
  [midje.util.laziness$eagerly invoke "laziness.clj" 5]
  [midje.parsing.1_to_explicit_form.facts$midjcoexpand invokeStatic "facts.clj" 124]
  [midje.parsing.1_to_explicit_form.facts$midjcoexpand invoke "facts.clj" 108]
  [clojure.core$map$fn__5885 invoke "core.clj" 2759]
  [clojure.lang.LazySeq sval "LazySeq.java" 42]
  [clojure.lang.LazySeq seq "LazySeq.java" 51]
  [clojure.lang.RT seq "RT.java" 535]
  [clojure.core$seq__5420 invokeStatic "core.clj" 139]
  [clojure.core$map$fn__5885 invoke "core.clj" 2750]
  [clojure.lang.LazySeq sval "LazySeq.java" 42]
  [clojure.lang.LazySeq seq "LazySeq.java" 51]
  [clojure.lang.Cons next "Cons.java" 39]
  [clojure.lang.PersistentList$Primordial doInvoke "PersistentList.java" 37]
  [clojure.lang.RestFn applyTo "RestFn.java" 137]
  [clojure.core$apply invokeStatic "core.clj" 667]
  [clojure.core$apply invoke "core.clj" 662]
  [midje.util.laziness$eagerly invokeStatic "laziness.clj" 15]
  [midje.util.laziness$eagerly invoke "laziness.clj" 5]
  [midje.parsing.1_to_explicit_form.facts$midjcoexpand invokeStatic "facts.clj" 124]
  [midje.parsing.1_to_explicit_form.facts$midjcoexpand invoke "facts.clj" 108]
  [midje.parsing.1_to_explicit_form.facts$expand_fact_body invokeStatic "facts.clj" 162]
  [midje.parsing.1_to_explicit_form.facts$expand_fact_body invoke "facts.clj" 154]
  [midje.parsing.1_to_explicit_form.facts$complete_fact_transformation invokeStatic "facts.clj" 200]
  [midje.parsing.1_to_explicit_form.facts$complete_fact_transformation invoke "facts.clj" 196]
  [midje.sweet$fact$fn__44887 invoke "sweet.clj" 194]
  [midje.parsing.util.error_handling$parse_and_catch_failure invokeStatic "error_handling.clj" 41]
  [midje.parsing.util.error_handling$parse_and_catch_failure invoke "error_handling.clj" 17]
  [midje.sweet$fact invokeStatic "sweet.clj" 186]
  [midje.sweet$fact doInvoke "sweet.clj" 170]
  [clojure.lang.RestFn applyTo "RestFn.java" 142]
  [clojure.lang.Var applyTo "Var.java" 705]
  [clojure.lang.Compiler macroexpand1 "Compiler.java" 6997]
  [clojure.lang.Compiler macroexpand "Compiler.java" 7079]
  [clojure.lang.Compiler eval "Compiler.java" 7165]
  [clojure.lang.Compiler eval "Compiler.java" 7136]
  [clojure.core$eval invokeStatic "core.clj" 3202]
  [clojure.core$eval invoke "core.clj" 3198]
  [nrepl.middleware.interruptible_eval$evaluate$fn__949$fn__950 invoke "interruptible_eval.clj" 87]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 667]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1977]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1977]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [nrepl.middleware.interruptible_eval$evaluate$fn__949 invoke "interruptible_eval.clj" 87]
  [clojure.main$repl$read_eval_print__9112$fn__9115 invoke "main.clj" 437]
  [clojure.main$repl$read_eval_print__9112 invoke "main.clj" 437]
  [clojure.main$repl$fn__9121 invoke "main.clj" 458]
  [clojure.main$repl invokeStatic "main.clj" 458]
  [clojure.main$repl doInvoke "main.clj" 368]
  [clojure.lang.RestFn applyTo "RestFn.java" 137]
  [clojure.core$apply invokeStatic "core.clj" 667]
  [clojure.core$apply invoke "core.clj" 662]
  [refactor_nrepl.ns.slam.hound.regrow$wrap_clojure_repl$fn__17903 doInvoke "regrow.clj" 20]
  [clojure.lang.RestFn invoke "RestFn.java" 1523]
  [nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 84]
  [nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 56]
  [nrepl.middleware.interruptible_eval$interruptible_eval$fn__980$fn__984 invoke "interruptible_eval.clj" 152]
  [clojure.lang.AFn run "AFn.java" 22]
  [nrepl.middleware.session$session_exec$main_loop__1048$fn__1052 invoke "session.clj" 202]
  [nrepl.middleware.session$session_exec$main_loop__1048 invoke "session.clj" 201]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 832]]}
vemv commented 3 years ago

@philomates As a humble suggestion, I wonder if it has to do with some ^:dynamic var; for example most (if not all) Clojure test runners assume that *ns* equals (the-ns 'user), while that's not necessarily the case in an IDE's repl.

I see there's a number of ^:dynamic vars in Midje.

genmeblog commented 3 years ago

I've created minimal project to test some of the paths. So:

deps.edn

{:deps {org.clojure/clojure {:mvn/version "1.10.3"}
        midje/midje {:mvn/version "1.9.9"}}}

project.clj

(defproject midjebug "0.1.0-SNAPSHOT"
  :plugins [[lein-tools-deps "0.4.5"]]
  :middleware [lein-tools-deps.plugin/resolve-dependencies-with-deps-edn]
  :lein-tools-deps/config {:config-files [:install :user :project]})

test/midjebug/core_test.clj

(ns midjebug.core-test
  (:require [midje.sweet :refer [fact tabular =>]]))

(fact "test midje bug under Cider"
  (tabular (fact ?l => ?r)
           ?l ?r
           1  1))

Versions

Leiningen 2.9.5 on Java 14.0.2 OpenJDK 64-Bit Server VM
CIDER 1.1.0snapshot (package: 20210302.849), nREPL 0.8.3
Clojure 1.10.3, Java 14.0.2
GNU Emacs 26.3 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.14) of 2020-03-26, modified by Debian
Ubuntu 20.04.2 LTS on WSL2 (Windows 10)
clj Version: 1.10.2.796

After cider-jack-in-clj in Emacs, when:

clojure-clj is used, everything works ok
;;  Startup: /usr/local/bin/clojure -Sdeps '{:deps {nrepl/nrepl {:mvn/version "0.8.3"} refactor-nrepl {:mvn/version "2.5.1"} com.billpiel/sayid {:mvn/version "0.1.0"} cider/cider-nrepl {:mvn/version "0.25.9"} nubank/midje-nrepl {:mvn/version "1.2.0-SNAPSHOT"}} :aliases {:cider/nrepl {:main-opts ["-m" "nrepl.cmdline" "--middleware" "[refactor-nrepl.middleware/wrap-refactor,com.billpiel.sayid.nrepl-middleware/wrap-sayid,cider.nrepl/cider-middleware]"]}}}' -M:cider/nrepl
lein is used, above exception is thrown
;;  Startup: /home/XXX/bin/lein update-in :dependencies conj \[nrepl/nrepl\ \"0.8.3\"\] -- update-in :plugins conj \[refactor-nrepl\ \"2.5.1\"\] -- update-in :plugins conj \[com.billpiel/sayid\ \"0.1.0\"\] -- update-in :plugins conj \[cider/cider-nrepl\ \"0.25.9\"\] -- update-in :plugins conj \[nubank/midje-nrepl\ \"1.2.0-SNAPSHOT\"\] -- repl :headless :host localhost

(~/.lein/profiles.clj file was removed for tests)

philomates commented 3 years ago

Thanks for the reproduction steps @genmeblog! I'll take a look at it this week and report back

philomates commented 3 years ago

okay, I installed spacemacs and spun this up. I think this is an issue with nubank/midje-repl. I had to include it in the project.clj to cause the aforementioned exception. I'm pretty unfamiliar with what is going on in that library. @alan-ghelardi can you help us out?

alan-ghelardi commented 3 years ago

@philomates midje-nrepl has a middleware that binds the dynamic var clojure.test/*load-tests* to false in order to prevent Midje from loading facts automatically when the namespace in question is loaded. Midje checks the value of this var on facts expansion to decide if it should run the test or create a var with the contents of the fact.

By binding this var I think that you will be able to reproduce the issue without depending on midje-nrepl.

@vemv if the issue is bothering you, you can disable the aforementioned behavior by running emidje-toggle-load-facts-on-eval in a buffer basis or disable the feature entirely by tweaking the var emidje-load-facts-on-eval, for instance, in your .dir-locals.el:

((nil . ((emidje-load-facts-on-eval . nil))))