Running a specific test suite in the REPL runs all test suites instead

[S4G4R]

I have a scenario where I have two different entrypoints in my application (two deployables). I want to test these separately. Here is my config file

{:kaocha/tests [{:kaocha.testable/type :kaocha.type/clojure.test
                 :kaocha.testable/id :suite-a
                 :kaocha/ns-patterns ["-test$"]
                 :kaocha/source-paths ["src/a"]
                 :kaocha/test-paths ["test/a"]}
                {:kaocha.testable/type :kaocha.type/clojure.test
                 :kaocha.testable/id :suite-b
                 :kaocha/ns-patterns ["-test$"]
                 :kaocha/source-paths ["src/b"]
                 :kaocha/test-paths ["test/b"]}]
 :kaocha/fail-fast? false
 :kaocha/color? true
 :kaocha/reporter [kaocha.report/dots]
 :kaocha/plugins [:kaocha.plugin/capture-output]}

This works as expected:

lein run -m kaocha.runner suite-a

But, if I try doing it from the REPL

(kaocha.repl/run :suite-a)

it seems to be running both test suites.

I was able to hack around it using this

(kaocha.repl/run (update (kaocha.repl/config)
                         :kaocha/tests
                         #(filter (fn [{suite-id :kaocha.testable/id}]
                                    (= suite-id :suite-a))
                                  %)))

It looks like kaocha.runner/-main does employ a similar strategy as my hack, but kaocha.repl/run does something completely different.

I'm not sure if my usage is wrong, but I'd also propose harmonizing both behaviours here.

[alysbrooks]

Some (if not the majority) of differences between different ways of invoking Kaocha are intentional, but I agree this particular difference is, at the very least, confusing. My guess is that this specific discrepancy is less due to the specific way kaocha.repl/run works and more likely a bug in --focus/:kaocha.filter/focus, since I would expect the two ways of filtering to behave identically here.

I noticed you linked to v1.85.1342. Is that the version of Kaocha you're using?

Thanks for reporting!

[S4G4R]

I'm actually using the latest version 1.87.1366. I'm unsure how it linked to v1.85.1342, apologies!

I'd be interested in debugging this and potentially fixing it. Are there any instructions somewhere on how to build and develop kaocha locally?

Coming from a Leiningen background, I'm used to a simple lein install and using checkout dependencies to work on projects in parallel.

[plexus]

If you're using clojure CLI then use :local/root instead of :mvn/version. That's all, there's no build step.

[S4G4R]

Thanks. I'm not really using the Clojure CLI, I would like to use it with my existing Leiningen project.clj. I suppose that's not really possible.

I will try using the Clojure CLI or perhaps create a dummy project.clj representation of the current deps.edn file.

[alysbrooks]

Are you wondering about how to use a development version of Kaocha with your project and Leiningen? I believe you can use Leiningen plugins to use Kaocha from a local Git repository. You could also install your development version of Kaocha into the local Maven cache (so Leiningen will pick that up instead of going to Clojars), but I don't have time right now to find or provide instructions. (I also think this is the messier of the two options.)

[S4G4R]

Well, I feel stupid. After looking around the codebase and trying to debug the problem, I realised I was missing :kaocha.plugin/filter in my tests.edn file.

:kaocha.plugins [:kaocha.plugin/filter]

It wasn't super clear from the docs that this is needed. I understand that it is included by default if you use the #kaocha/v1 {} reader literal.

https://cljdoc.org/d/lambdaisland/kaocha/1.87.1366/doc/3-configuration#plugins

[alysbrooks]

Honestly, the kaocha.repl/run should probably warn the user if that plugin isn't available, since it using filter for this functionality is an implementation detail in my opinion.