how to run tests in the shadow-cljs project

[zcaudate]

Are there instructions to build and run the different tests defined here: https://github.com/thheller/shadow-cljs/blob/master/shadow-cljs.edn?

[thheller]

lein test is pretty much the only "test" right now. I use the test builds in shadow-cljs.edn occasionally to do some manual testing but nothing automated exists for that.

[zcaudate]

I'm looking to get CI setup for a test library I'm working on.

It's currently working in repl by calling yin.test/run (a macro)

Here are the two test files: https://github.com/zcaudate/yin/blob/master/test/yin/test/common/primitives_test.cljs https://github.com/zcaudate/yin/blob/master/test/yin/test/checker/base_test.cljs

How would I get configure shadow-cljs to output something that runs via node?

[thheller]

https://shadow-cljs.github.io/docs/UsersGuide.html#target-node-test

[zcaudate]

it built, but it's not what I want. It's running the default clojure tests.

I need it to run a main that I'll have to define myself.

[zcaudate]

okay don't worry: https://shadow-cljs.github.io/docs/UsersGuide.html#target-node-script

[thheller]

The test targets typically allow you to configure a :runner-ns. For :node-test it is currently hardcoded to https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/test/node.cljs but I could make that configurable as well.

But :node-script works as well. The only added thing the :node-test target does is look for namespaces by regexp so you don't have to manually require all the tests.

If a configurable :runner-ns would work for you let me know.

[zcaudate]

I'm getting this error:

it has something to do with:

SHADOW_IMPORT("shadow.js.shim.module$ws.js");

the configuration is:

:test {:target    :node-script
       :main      yin.dev/-main
       :output-to "out/node-tests.js"}

[thheller]

Cannot find module 'ws' is node-speak for "the ws package is not installed". It is installed automatically if you npm install shadow-cljs in your project.

The ws package provides a websocket for node which is used to connect to the dev server for live reloading stuff.

[zcaudate]

I've gone back to the test runner because it seems to at least be able to run the tests. I would like to be able to type (run) and run everything but it's not working. The problem I'm having is that the vars are being wiped and I need to explicitly add the tests to the ns :require block, ie:

(ns yin.dev-test
  (:require-macros [yin.test :refer [run]])
  (:require [yin.common.primitives-test
             yin.test.checker.base-test]))

(run)

This works, but it runs the tests twice because by default, fact blocks are functions that get evaluated immediately. So the tests run once when they are required and once when run is called.

I had a look at cljs.test and saw that it is also using find-ns and all-ns to resolve test files. In yin.test, fact is also defined as a defn:

https://github.com/zcaudate/yin/blob/master/src/yin/test.clj#L51-L71

but for whatever reason, when there is a call to cljs.analyzer.api/find-ns and cljs.analyzer.api/all-ns, the vars disappear, ie. run-namespace:

https://github.com/zcaudate/yin/blob/master/src/yin/test.clj#L85

is there anything I'm missing

[thheller]

No idea how you test frameworks works and no time to look into it.

I also don't understand your description completely. What is the problem exactly? You say tests run twice? fact runs immediately and (run) runs them again so that is expected no?

[zcaudate]

So the whole this kinda works:

(ns yin.dev-test
  (:require-macros [yin.test :refer [run require-tests]])
  (:require [yin.common.primitives-test]
            [yin.test.checker.base-test]
            [yin.test.runner-test]))

(run)

the first bit comes from the initial require

the second bit comes from the (run)

---

if the requires are not there, then this happens - I'm assuming because the tests haven't been compiled before (run) is called:

[thheller]

FWIW do NOT go the way cljs.test went and make everything macros. This just makes everything harder and more annoying to work with. If you instead "register" the test you can skip the entire find-ns and all-ns shenanigans.

I hack this into cljs.test here https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/build/cljs_hacks.cljc#L818

But you control the fact macro so just do edit the macro to emit (yin.test/register-fact #'the-fact) and maintain the registry yourself instead of relying on the analyzer metadata via macros.

IMHO, YMMV.

[thheller]

Yes, if the namespaces are not required anywhere they are not compiled. The special exception for this are the test builds but everything else follows this strictly.

See https://code.thheller.com/blog/shadow-cljs/2018/02/08/problem-solved-source-paths.html

[zcaudate]

emit (yin.test/register-fact #'the-fact)

brilliant

special exception for this are the test builds

Is this a cljs.test thing? is there any way to:

1. compile and load the test file with an runtime flag eval set to false (for suppressing immediate call)
2. (run) will then just work

[zcaudate]

also, regarding customising the test runner, run returns a map:

(run) ;;=> {:files 2, :thrown 0, :facts 9, :checks 24, :passed 24, :failed 0}

how can this be hooked into the build system to return a failed/pass? In clojure this is pretty easy https://github.com/zcaudate/hara/blob/master/src/hara/test.clj#L71-L80

something like this: https://travis-ci.org/zcaudate/hara/jobs/392203238

[thheller]

I updated the :node-test target to allow a custom :main in 2.4.14.

You could then use {:target :node-test :main yin.test/main ...} in your config and then

(defn main []
  (let [result (run)]
    (when-not (success? result)
      (js/process.exit 1))))

The exit code is not propagated when using :autorun true so you'd need to shadow-cljs compile tests && node out/tests.js.

I do not recommend running anything when the namespace itself is loaded. If you instead run everything in main you avoid situations where an exception while loading code blows everything up.

Thas basically what the shadow.test.node does. https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/test/node.cljs#L8-L14

[thheller]

The :node-test target is responsible for adding all the namespaces that match the given regexp to the build. It is not a cljs.test thing.

[zcaudate]

great. I'll check it out.

Is it possible to set a value somewhere in the compilation process so that I can use it to disable evaluation?

something like:

{:target    :node-test
  :output-to "out/node-tests.js"
  :set {yin.test/*compilation* true}}

or

{:target    :node-test
  :output-to "out/node-tests.js"
  :env "COMPILE"}

Then I can access it in the fact macro on compile time.

I really don't want to lose the ability to evaluate tests in repl. Like I'd rather it run twice on an automated test because being able to see the result straight away is critical for my dev flow.

[thheller]

https://shadow-cljs.github.io/docs/UsersGuide.html#closure-defines

[zcaudate]

@thheller: done. thanks for all your help.

https://travis-ci.org/zcaudate/yin/jobs/399168603