search

lambdaisland / kaocha

Full featured next gen Clojure test runner
https://cljdoc.org/d/lambdaisland/kaocha/
Eclipse Public License 1.0
803 stars 84 forks source link

Kaocha crashes if test assertion runs after deftest function #384

Closed vincentjames501 closed 1 year ago

vincentjames501 commented 1 year ago

We have a use-case where we have a fixture that does some final assertions after the test runs. Looks like a recent update to Kaocha changed some behavior here.

Here is a simple failure example:

(ns kaocha-failure-test
  (:require [clojure.test :refer :all]))

(defn- post-assertion-fixture
  [f]
  (f)
  (is (= true true)))

(use-fixtures :each post-assertion-fixture)

(deftest my-test
  (is (= 2 (+ 1 1))))
$ ./kaocha --focus kaocha-failure-test
...
[(..Execution error (ClassCastException) at kaocha.testable/run (testable.clj:131).
class java.lang.Boolean cannot be cast to class clojure.lang.Associative (java.lang.Boolean is in module java.base of loader 'bootstrap'; clojure.lang.Associative is in unnamed module of loader 'app')

Full report at:
/var/folders/wn/cfyvfy_j60n7rzr9ndvgwtcm0000gp/T/clojure-8244869028492603707.edn
Error encountered performing task 'run' with profile(s): 'base,system,user,provided,dev'
Suppressed exit

$ cat /var/folders/wn/cfyvfy_j60n7rzr9ndvgwtcm0000gp/T/clojure-8244869028492603707.edn
{:clojure.main/message
 "Execution error (ClassCastException) at kaocha.testable/run (testable.clj:131).\nclass java.lang.Boolean cannot be cast to class clojure.lang.Associative (java.lang.Boolean is in module java.base of loader 'bootstrap'; clojure.lang.Associative is in unnamed module of loader 'app')\n",
 :clojure.main/triage
 {:clojure.error/class java.lang.ClassCastException,
  :clojure.error/line 131,
  :clojure.error/cause
  "class java.lang.Boolean cannot be cast to class clojure.lang.Associative (java.lang.Boolean is in module java.base of loader 'bootstrap'; clojure.lang.Associative is in unnamed module of loader 'app')",
  :clojure.error/symbol kaocha.testable/run,
  :clojure.error/source "testable.clj",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type clojure.lang.Compiler$CompilerException,
    :message
    "Syntax error macroexpanding at (/private/var/folders/wn/cfyvfy_j60n7rzr9ndvgwtcm0000gp/T/form-init10811224181315925298.clj:1:126).",
    :data
    {:clojure.error/phase :execution,
     :clojure.error/line 1,
     :clojure.error/column 126,
     :clojure.error/source
     "/private/var/folders/wn/cfyvfy_j60n7rzr9ndvgwtcm0000gp/T/form-init10811224181315925298.clj"},
    :at [clojure.lang.Compiler load "Compiler.java" 7665]}
   {:type java.lang.ClassCastException,
    :message
    "class java.lang.Boolean cannot be cast to class clojure.lang.Associative (java.lang.Boolean is in module java.base of loader 'bootstrap'; clojure.lang.Associative is in unnamed module of loader 'app')",
    :at [clojure.lang.RT assoc "RT.java" 827]}],
  :trace
  [[clojure.lang.RT assoc "RT.java" 827]
   [clojure.core$assoc__5481 invokeStatic "core.clj" 193]
   [clojure.core$assoc__5481 invoke "core.clj" 192]
   [kaocha.testable$run invokeStatic "testable.clj" 131]
   [kaocha.testable$run invoke "testable.clj" 120]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.spec.test.alpha$spec_checking_fn$fn__3023
    doInvoke
    "alpha.clj"
    153]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [kaocha.testable$run_testable invokeStatic "testable.clj" 211]
   [kaocha.testable$run_testable invoke "testable.clj" 158]
   [kaocha.testable$run_testables invokeStatic "testable.clj" 224]
   [kaocha.testable$run_testables invoke "testable.clj" 214]
   [clojure.lang.Atom swap "Atom.java" 51]
   [clojure.core$swap_BANG_ invokeStatic "core.clj" 2370]
   [clojure.core$swap_BANG_ invoke "core.clj" 2362]
   [kaocha.type.ns$run_tests$fn__55981 invoke "ns.clj" 21]
   [clojure.test$default_fixture invokeStatic "test.clj" 687]
   [clojure.test$default_fixture invoke "test.clj" 683]
   [kaocha.type.ns$run_tests invokeStatic "ns.clj" 21]
   [kaocha.type.ns$run_tests invoke "ns.clj" 17]
   [kaocha.type.ns$eval55999$fn__56000 invoke "ns.clj" 60]
   [clojure.lang.MultiFn invoke "MultiFn.java" 234]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$apply invoke "core.clj" 662]
   [kaocha.plugin.capture_output$capture_output_wrap_run_hook$fn__3441$fn__3442
    invoke
    "capture_output.cljc"
    88]
   [clojure.core$with_redefs_fn invokeStatic "core.clj" 7582]
   [clojure.core$with_redefs_fn invoke "core.clj" 7566]
   [kaocha.plugin.capture_output$capture_output_wrap_run_hook$fn__3441
    doInvoke
    "capture_output.cljc"
    88]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [kaocha.testable$run invokeStatic "testable.clj" 129]
   [kaocha.testable$run invoke "testable.clj" 120]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.spec.test.alpha$spec_checking_fn$fn__3023
    doInvoke
    "alpha.clj"
    153]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [kaocha.testable$run_testable invokeStatic "testable.clj" 211]
   [kaocha.testable$run_testable invoke "testable.clj" 158]
   [kaocha.testable$run_testables invokeStatic "testable.clj" 224]
   [kaocha.testable$run_testables invoke "testable.clj" 214]
   [kaocha.test_suite$run invokeStatic "test_suite.clj" 7]
   [kaocha.test_suite$run invoke "test_suite.clj" 5]
   [kaocha.type.clojure.test$eval56501$fn__56502 invoke "test.clj" 21]
   [clojure.lang.MultiFn invoke "MultiFn.java" 234]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$apply invoke "core.clj" 662]
   [kaocha.plugin.capture_output$capture_output_wrap_run_hook$fn__3441$fn__3442
    invoke
    "capture_output.cljc"
    88]
   [clojure.core$with_redefs_fn invokeStatic "core.clj" 7582]
   [clojure.core$with_redefs_fn invoke "core.clj" 7566]
   [kaocha.plugin.capture_output$capture_output_wrap_run_hook$fn__3441
    doInvoke
    "capture_output.cljc"
    88]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [kaocha.testable$run invokeStatic "testable.clj" 129]
   [kaocha.testable$run invoke "testable.clj" 120]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.spec.test.alpha$spec_checking_fn$fn__3023
    doInvoke
    "alpha.clj"
    153]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [kaocha.testable$run_testable invokeStatic "testable.clj" 211]
   [kaocha.testable$run_testable invoke "testable.clj" 158]
   [kaocha.testable$run_testables invokeStatic "testable.clj" 224]
   [kaocha.testable$run_testables invoke "testable.clj" 214]
   [kaocha.api$run$fn__4005$fn__4009$fn__4010 invoke "api.clj" 148]
   [clojure.core$with_redefs_fn invokeStatic "core.clj" 7582]
   [clojure.core$with_redefs_fn invoke "core.clj" 7566]
   [kaocha.api$run$fn__4005$fn__4009 invoke "api.clj" 122]
   [clojure.lang.AFn applyToHelper "AFn.java" 152]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
   [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
   [clojure.lang.RestFn invoke "RestFn.java" 425]
   [kaocha.api$run$fn__4005 invoke "api.clj" 121]
   [clojure.lang.AFn applyToHelper "AFn.java" 152]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
   [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
   [clojure.lang.RestFn invoke "RestFn.java" 425]
   [kaocha.api$run invokeStatic "api.clj" 99]
   [kaocha.api$run invoke "api.clj" 86]
   [kaocha.plugin.cloverage$eval8766$fn__8767$fn__8768
    invoke
    "cloverage.clj"
    191]
   [cloverage.coverage$run_tests$fn__8714 invoke "coverage.clj" 257]
   [cloverage.coverage$form_for_suppressing_unselected_tests
    invokeStatic
    "coverage.clj"
    54]
   [cloverage.coverage$form_for_suppressing_unselected_tests
    invoke
    "coverage.clj"
    32]
   [cloverage.coverage$run_tests invokeStatic "coverage.clj" 255]
   [cloverage.coverage$run_tests invoke "coverage.clj" 245]
   [cloverage.coverage$run_main invokeStatic "coverage.clj" 307]
   [cloverage.coverage$run_main invoke "coverage.clj" 299]
   [kaocha.plugin.cloverage$run_cloverage
    invokeStatic
    "cloverage.clj"
    152]
   [kaocha.plugin.cloverage$run_cloverage invoke "cloverage.clj" 139]
   [kaocha.plugin.cloverage$cloverage_main_hook
    invokeStatic
    "cloverage.clj"
    185]
   [kaocha.plugin.cloverage$cloverage_main_hook
    invoke
    "cloverage.clj"
    154]
   [clojure.lang.AFn applyToHelper "AFn.java" 154]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.core$apply invokeStatic "core.clj" 669]
   [clojure.core$apply invoke "core.clj" 662]
   [kaocha.plugin$run_hook_STAR_$fn__2191 invoke "plugin.clj" 91]
   [clojure.lang.PersistentVector reduce "PersistentVector.java" 343]
   [clojure.core$reduce invokeStatic "core.clj" 6885]
   [clojure.core$reduce invoke "core.clj" 6868]
   [kaocha.plugin$run_hook_STAR_ invokeStatic "plugin.clj" 89]
   [kaocha.plugin$run_hook_STAR_ doInvoke "plugin.clj" 88]
   [clojure.lang.RestFn invoke "RestFn.java" 445]
   [clojure.lang.AFn applyToHelper "AFn.java" 160]
   [clojure.lang.RestFn applyTo "RestFn.java" 132]
   [clojure.core$apply invokeStatic "core.clj" 673]
   [clojure.core$apply invoke "core.clj" 662]
   [kaocha.plugin$run_hook invokeStatic "plugin.clj" 100]
   [kaocha.plugin$run_hook doInvoke "plugin.clj" 99]
   [clojure.lang.RestFn invoke "RestFn.java" 425]
   [kaocha.runner$run$fn__4068 invoke "runner.clj" 131]
   [clojure.lang.AFn applyToHelper "AFn.java" 152]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
   [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
   [clojure.lang.RestFn invoke "RestFn.java" 425]
   [kaocha.runner$run invokeStatic "runner.clj" 130]
   [kaocha.runner$run invoke "runner.clj" 73]
   [kaocha.runner$_main_STAR_ invokeStatic "runner.clj" 176]
   [kaocha.runner$_main_STAR_ doInvoke "runner.clj" 144]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$apply invoke "core.clj" 662]
   [kaocha.runner$_main invokeStatic "runner.clj" 187]
   [kaocha.runner$_main doInvoke "runner.clj" 185]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [clojure.lang.Var invoke "Var.java" 388]
   [user$eval140 invokeStatic "form-init10811224181315925298.clj" 1]
   [user$eval140 invoke "form-init10811224181315925298.clj" 1]
   [clojure.lang.Compiler eval "Compiler.java" 7194]
   [clojure.lang.Compiler eval "Compiler.java" 7184]
   [clojure.lang.Compiler load "Compiler.java" 7653]
   [clojure.lang.Compiler loadFile "Compiler.java" 7591]
   [clojure.main$load_script invokeStatic "main.clj" 475]
   [clojure.main$init_opt invokeStatic "main.clj" 477]
   [clojure.main$init_opt invoke "main.clj" 477]
   [clojure.main$initialize invokeStatic "main.clj" 508]
   [clojure.main$null_opt invokeStatic "main.clj" 542]
   [clojure.main$null_opt invoke "main.clj" 539]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause
  "class java.lang.Boolean cannot be cast to class clojure.lang.Associative (java.lang.Boolean is in module java.base of loader 'bootstrap'; clojure.lang.Associative is in unnamed module of loader 'app')",
  :phase :execution}}
plexus commented 1 year ago

Thanks for the report, @vincentjames501 . We'll have a look. @NoahTheDuke do you think this could be related to https://github.com/lambdaisland/kaocha/pull/368 ?

NoahTheDuke commented 1 year ago

It might be. I'll take a look today, I don't want anything I contributed breaking existing code.

NoahTheDuke commented 1 year ago

Yes, it's because of #368. I'm wrapping up a fix in a PR right now. My sincerest apologies, @vincentjames501 and @plexus.

alysbrooks commented 1 year ago

Thanks for the quick fix, @NoahTheDuke! I can take a look at it today and maybe release it right away if it's straightforward enough.

vincentjames501 commented 1 year ago

@NoahTheDuke no need to apologize! Awesome work on the quick fix. I love the open-source community!

plexus commented 1 year ago

Thanks a lot @NoahTheDuke ! Great work!