lambdaisland / kaocha-cljs

ClojureScript support for Kaocha
Eclipse Public License 1.0
40 stars 9 forks source link

Shadow-CLJS integration #2

Closed orestis closed 1 year ago

orestis commented 5 years ago

Hi! Thanks for Kaocha, it’s very friendly and finally our Travis CI logs make sense :slightly_smiling_face:

I was looking at Kaocha-cljs, but I’d like to integrate it with shadow-cljs somehow, to run node unit tests. Currently shadow has its own test runner, which looks like this: https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/test/node.cljs

The main difference is that shadow runs the tests always in a CLJS mode, so things like pretty-printing diffs etc have to be CLJS compatible… which I’m not sure Kaocha handles? It looks like it reports the test results via a websocket somehow?

It seems to me that shadow-cljs gives you many hooks during the compilation process, so this should be possible. I'll ask around and see.

orestis commented 5 years ago

A little bit more info after discussing with @thheller:

One important design decision behind shadow-cljs test running is to run the tests against compiled builds, instead of REPL-evaluated code. So given that requirement, I wonder if kaocha-cljs can somehow work against an already-compiled .js file.

plexus commented 5 years ago

Kaocha works differently from most ClojureScript test runners in that it works against a REPL backend, rather than first compiling a single big JS blob and running that. This is quite deliberate, because of the dynamic nature of Kaocha. We send commands to the REPL to start tests one by one, then use a websocket to send cljs.test events back to Kaocha and route them to the kaocha reporter.

So to make this work in a shadow-cljs project you'd need a cljs repl-env that uses the shadow-cljs compiler, so that npm references are dealt with the shadow-cljs way. You can configure :cljs/repl-env in your ClojureScript test suite to point to a function which returns a repl-env, for example

#kaocha/v1
{:tests [{:type :kaocha.type/cljs
          :cljs/repl-env cljs.repl.node/repl-env}]}

So if you or @thheller can come up with a function that returns a shadow-cljs backed repl-env then Kaocha should work with that just fine.

plexus commented 5 years ago

I chatted some more about this with @thheller. The current kaocha-cljs implementation relies on the ClojureScript prepl interface, which is apparently not flexible enough to easily allow hooking in shadow-cljs, however Thomas might provide a similar API that we can hook into, or create a derived Kaocha test type specifically for Shadow-cljs.

So there will likely be a solution at some point, until then you're best off using Shadow-cljs's own test runner capabilities.

lins05 commented 4 years ago

Upvote for this ... Especially shadow-cljs is more and more commonly chosen as the build tool for new cljs projects these days

plexus commented 4 years ago

Saying "upvote" is about as useful as me saying "Pull request welcome". Add a :+1: reaction to the original post if you're interested in this, or help to put in the work to make it happen. I'm available for pairing sessions if people want help understanding the Kaocha code base.

Someone needs to develop a repl-env or prepl compatible interface for shadow-cljs for this to happen. Until then kaocha-cljs has no way of interfacing with shadow.

zharinov commented 4 years ago

@plexus I wonder if current pREPL implementation is well enough to hook up with kaocha-cljs? There are some bugs, but it looks like something to work with. I would like to help with this btw.

thheller commented 4 years ago

FWIW the last time I looked at this kaocha complected compiling tests with running them. So it isn't possible to run tests in :advanced for example, which is something I would want when running tests.

So what would make sense for interop with shadow-cljs is a "runner" ns. Basically this would "register" all tests and on startup connect to the websocket kaocha provides. It can then "negotiate" which tests to run at runtime over that websocket (instead of the REPL) and actually run them without any further compilation. The hot-reload and REPL could be an optional layer on top though. The core "runner" ns would need nothing specific from shadow-cljs and would work with any other environment too.

As far as I can tell the only difference would be that kaocha really only acts as a runner and doesn't need to deal with compilation at all. That could be an optional addition if needed but in theory it should be enough to point kaocha at a .js file to do its stuff.

plexus commented 4 years ago

The suggestions by @zharinov and @thheller are both worth exploring. Please ping me if you decide to work on this and we can set up a time to pair.

thheller commented 4 years ago

@plexus are there any docs about custom runner types like this one for kaocha? It seems easier to me to just create a custom shadow-cljs runner than trying to make this work with it? I can try to make out which functions/events this is supposed to provide but docs would help a lot.

plexus commented 4 years ago

Writing your own test types is documented here

alysbrooks commented 1 year ago

Kaocha-cljs2 is going to be our focus moving forward, but we'd welcome a PR that improves integration between Shadow-CLJS and Kaocha-cljs.