search

lambdaisland / kaocha

Full featured next gen Clojure test runner
https://cljdoc.org/d/lambdaisland/kaocha/1.0.861/doc/1-introduction
Eclipse Public License 1.0
795 stars 82 forks source link

Filter out all kaocha-related stacktrace elements by default #58

Closed tonsky closed 11 months ago

tonsky commented 5 years ago

When test blows up with unexpected exception:

ERROR in datascript.test.upsert/test-redefining-ids (db.cljc:1083)
Uncaught exception, not in assertion.
Exception: clojure.lang.ExceptionInfo: Conflicting upsert: -3 resolves both to 1 and 2
{:error :transact/upsert}
 at datascript.db$retry_with_tempid.invokeStatic (db.cljc:1083)
    datascript.db$retry_with_tempid.invoke (db.cljc:1081)
    datascript.db$transact_tx_data.invokeStatic (db.cljc:1142)
    datascript.db$transact_tx_data.invoke (db.cljc:1103)
    datascript.db$retry_with_tempid.invokeStatic (db.cljc:1091)
    datascript.db$retry_with_tempid.invoke (db.cljc:1081)
    datascript.db$transact_tx_data.invokeStatic (db.cljc:1142)
    datascript.db$transact_tx_data.invoke (db.cljc:1103)
    datascript.core$with.invokeStatic (core.cljc:229)
    datascript.core$with.invoke (core.cljc:222)
    datascript.core$with.invokeStatic (core.cljc:224)
    datascript.core$with.invoke (core.cljc:222)
    datascript.core$db_with.invokeStatic (core.cljc:241)
    datascript.core$db_with.invoke (core.cljc:237)
    datascript.test.upsert$fn__11538.invokeStatic (upsert.cljc:149)
    datascript.test.upsert/fn (upsert.cljc:138)
    kaocha.plugin.capture_output$eval2423$fn__2424$fn__2437$fn__2439$fn__2440.invoke (capture_output.clj:92)
    kaocha.type.var$eval3337$fn__3339$fn__3344.invoke (var.clj:33)
    kaocha.type.var$eval3337$fn__3339.invoke (var.clj:30)
    ...
    kaocha.plugin.capture_output$eval2423$fn__2424$fn__2429$fn__2430$fn__2431.invoke (capture_output.clj:83)
    ...
    kaocha.plugin.capture_output$eval2423$fn__2424$fn__2429$fn__2430.doInvoke (capture_output.clj:83)
    ...
    kaocha.testable$run.invokeStatic (testable.clj:108)
    kaocha.testable$run.invoke (testable.clj:99)
    ...
    orchestra.spec.test$spec_checking_fn$fn__3103.doInvoke (test.clj:125)
    ...
    kaocha.testable$run_testable.invokeStatic (testable.clj:158)
    kaocha.testable$run_testable.invoke (testable.clj:137)
    kaocha.testable$run_testables.invokeStatic (testable.clj:167)
    kaocha.testable$run_testables.invoke (testable.clj:161)
    ...
    kaocha.type.ns$run_tests$fn__3403.invoke (ns.clj:57)
    ...
    kaocha.type.ns$run_tests.invokeStatic (ns.clj:57)
    kaocha.type.ns$run_tests.invoke (ns.clj:53)
    kaocha.type.ns$eval3407$fn__3408.invoke (ns.clj:75)
    ...
    kaocha.plugin.capture_output$eval2423$fn__2424$fn__2429$fn__2430$fn__2431.invoke (capture_output.clj:83)
    ...
    kaocha.plugin.capture_output$eval2423$fn__2424$fn__2429$fn__2430.doInvoke (capture_output.clj:83)
    ...
    kaocha.testable$run.invokeStatic (testable.clj:108)
    kaocha.testable$run.invoke (testable.clj:99)
    ...
    orchestra.spec.test$spec_checking_fn$fn__3103.doInvoke (test.clj:125)
    ...
    kaocha.testable$run_testable.invokeStatic (testable.clj:158)
    kaocha.testable$run_testable.invoke (testable.clj:137)
    kaocha.testable$run_testables.invokeStatic (testable.clj:167)
    kaocha.testable$run_testables.invoke (testable.clj:161)
    kaocha.type.clojure.test$eval3958$fn__3959.invoke (test.clj:17)
    ...
    kaocha.plugin.capture_output$eval2423$fn__2424$fn__2429$fn__2430$fn__2431.invoke (capture_output.clj:83)
    ...
    kaocha.plugin.capture_output$eval2423$fn__2424$fn__2429$fn__2430.doInvoke (capture_output.clj:83)
    ...
    kaocha.testable$run.invokeStatic (testable.clj:108)
    kaocha.testable$run.invoke (testable.clj:99)
    ...
    orchestra.spec.test$spec_checking_fn$fn__3103.doInvoke (test.clj:125)
    ...
    kaocha.testable$run_testable.invokeStatic (testable.clj:158)
    kaocha.testable$run_testable.invoke (testable.clj:137)
    kaocha.testable$run_testables.invokeStatic (testable.clj:167)
    kaocha.testable$run_testables.invoke (testable.clj:161)
    kaocha.api$run$fn__3051.invoke (api.clj:95)
    ...
    kaocha.api$run.invokeStatic (api.clj:87)
    kaocha.api$run.invoke (api.clj:70)
    kaocha.runner$run.invokeStatic (runner.clj:117)
    kaocha.runner$run.invoke (runner.clj:68)
    kaocha.runner$_main_STAR_.invokeStatic (runner.clj:135)
    kaocha.runner$_main_STAR_.doInvoke (runner.clj:121)
    ...
    kaocha.runner$_main.invokeStatic (runner.clj:146)
    kaocha.runner$_main.doInvoke (runner.clj:144)
    ...
    clojure.main$main_opt.invokeStatic (main.clj:491)
    clojure.main$main_opt.invoke (main.clj:487)
    clojure.main$main.invokeStatic (main.clj:598)
    clojure.main$main.doInvoke (main.clj:561)
    ...
    clojure.main.main (main.java:37)

Here anything starting with kaocha., clojure.main and orchestra. is not informative and actually obscures interesting parts of test output. Maybe filter them out by default?

tonsky commented 5 years ago

I want to mention here that current solution based on stacktrace element filtering is not a good idea to solve this. E.g. if I use orchestra in my code I don’t want it to be filtered out, but I do want the part of the stacktrace related to kaocha use of orchestra to be filtered out. Same goes for clojure.core

What I think implementation should look like:

seanpoulter commented 4 years ago

I've found this to be a pain point as well. Would you welcome a PR using @tonsky's proposed approach? If so, any tips on where to look for the stack trace output?

pesterhazy commented 1 year ago

Made a stab at this https://github.com/lambdaisland/kaocha/pull/321