Shadow-CLJS integration

[orestis]

Hi! Thanks for Kaocha, it’s very friendly and finally our Travis CI logs make sense :slightly_smiling_face:

I was looking at Kaocha-cljs, but I’d like to integrate it with shadow-cljs somehow, to run node unit tests. Currently shadow has its own test runner, which looks like this: https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/test/node.cljs

The main difference is that shadow runs the tests always in a CLJS mode, so things like pretty-printing diffs etc have to be CLJS compatible… which I’m not sure Kaocha handles? It looks like it reports the test results via a websocket somehow?

It seems to me that shadow-cljs gives you many hooks during the compilation process, so this should be possible. I'll ask around and see.

[orestis]

A little bit more info after discussing with @thheller:

One important design decision behind shadow-cljs test running is to run the tests against compiled builds, instead of REPL-evaluated code. So given that requirement, I wonder if kaocha-cljs can somehow work against an already-compiled .js file.

[plexus]

Kaocha works differently from most ClojureScript test runners in that it works against a REPL backend, rather than first compiling a single big JS blob and running that. This is quite deliberate, because of the dynamic nature of Kaocha. We send commands to the REPL to start tests one by one, then use a websocket to send cljs.test events back to Kaocha and route them to the kaocha reporter.

So to make this work in a shadow-cljs project you'd need a cljs repl-env that uses the shadow-cljs compiler, so that npm references are dealt with the shadow-cljs way. You can configure :cljs/repl-env in your ClojureScript test suite to point to a function which returns a repl-env, for example

#kaocha/v1
{:tests [{:type :kaocha.type/cljs
          :cljs/repl-env cljs.repl.node/repl-env}]}

So if you or @thheller can come up with a function that returns a shadow-cljs backed repl-env then Kaocha should work with that just fine.

[plexus]

I chatted some more about this with @thheller. The current kaocha-cljs implementation relies on the ClojureScript prepl interface, which is apparently not flexible enough to easily allow hooking in shadow-cljs, however Thomas might provide a similar API that we can hook into, or create a derived Kaocha test type specifically for Shadow-cljs.

So there will likely be a solution at some point, until then you're best off using Shadow-cljs's own test runner capabilities.

[lins05]

Upvote for this ... Especially shadow-cljs is more and more commonly chosen as the build tool for new cljs projects these days

[plexus]

Saying "upvote" is about as useful as me saying "Pull request welcome". Add a :+1: reaction to the original post if you're interested in this, or help to put in the work to make it happen. I'm available for pairing sessions if people want help understanding the Kaocha code base.

Someone needs to develop a repl-env or prepl compatible interface for shadow-cljs for this to happen. Until then kaocha-cljs has no way of interfacing with shadow.

[zharinov]

@plexus I wonder if current pREPL implementation is well enough to hook up with kaocha-cljs? There are some bugs, but it looks like something to work with. I would like to help with this btw.

[thheller]

FWIW the last time I looked at this kaocha complected compiling tests with running them. So it isn't possible to run tests in :advanced for example, which is something I would want when running tests.

So what would make sense for interop with shadow-cljs is a "runner" ns. Basically this would "register" all tests and on startup connect to the websocket kaocha provides. It can then "negotiate" which tests to run at runtime over that websocket (instead of the REPL) and actually run them without any further compilation. The hot-reload and REPL could be an optional layer on top though. The core "runner" ns would need nothing specific from shadow-cljs and would work with any other environment too.

As far as I can tell the only difference would be that kaocha really only acts as a runner and doesn't need to deal with compilation at all. That could be an optional addition if needed but in theory it should be enough to point kaocha at a .js file to do its stuff.

[plexus]

The suggestions by @zharinov and @thheller are both worth exploring. Please ping me if you decide to work on this and we can set up a time to pair.

[thheller]

@plexus are there any docs about custom runner types like this one for kaocha? It seems easier to me to just create a custom shadow-cljs runner than trying to make this work with it? I can try to make out which functions/events this is supposed to provide but docs would help a lot.

[plexus]

Writing your own test types is documented here

[alysbrooks]

Kaocha-cljs2 is going to be our focus moving forward, but we'd welcome a PR that improves integration between Shadow-CLJS and Kaocha-cljs.