Browser test runner(s)

[crisptrutski]

Interested in adding this? Thinking a QUnit style "refresh and read it" to start with, but this could be extended to drive Karma or a custom reporter to gather those stats back in the console.

[bensu]

I don't have any experience with Karma or QUnit but I'll check them out.

I have thought about this but haven't decided. My issue is with the page reloading, bringing something like Figwheel in (or any browser coordination) would make doo very big. My only idea so far is to offload everything to devcards

[crisptrutski]

Devcard tests are pretty nifty, but unless you're talking about generating the card code itself, not much missing there :smile:

Re: QUnit, really just suggesting something as basic as this: https://github.com/mike-thompson-day8/cljsbuild-none-test-seed/blob/master/test.html#L69-L79

Perhaps all that's needed here is support for custom commands. Fancier runners can be separate tasks that just pick up the js from the file(system / set).

[bensu]

@crisptrutski I got Karma working with Chrome, Firefox, and Opera. We are entering the npm land and I need to make some API decisions. Do you already have a package.json in your files or would you rather avoid that?

[crisptrutski]

@bensu occasionally I use jspm when using javascript directly off github releases, which introduces an explicit package.json file. Use of lein-npm is quite prevalent for vanilla npm usage - it is even included in mies. This then takes the approach of generating the package.json.

I think being tolerant but agnostic would be ideal - generate new or amend existing package.json to add necessary deps (or respect their existing versions), and if there are options to set in package.json for karma, allow those to be set easily in project.clj (and I'll track that for build.boot).

Not sure how tricky it would be in leiningen to make those modifications on top of a lein-npm generated package.json. Alternately if you want to just require a real package.json, we could submit a PR to https://github.com/RyanMcG/lein-npm/issues/23 to extend reach.

[bensu]

It's hard to deal with a config file that can be both machine generated and user edited. From what I gather the best approach is to document and allow three possibilities:

• Easy global installation through npm install -g karma (requires a flag).
• Indirect config through lein-npm (requires figuring it out).
• Manual config of package.json by the user (requires good docs).

I'll keep you posted. Thanks for the feedback!

[crisptrutski]

No problem, and good call on avoiding avoidable magic :star2: :zap:

[bensu]

I have just deployed 0.1.5-SNAPSHOT with Karma support and updated docs. @crisptrutski do you want to give it a spin before we close this issue? It would be great if you could test it with Safari!

[crisptrutski]

Hey Sebastian,

Awesome that you've put so much thought and work into this - a really great feature :smile:

Was able to get this working with some hacking around, but it didn't work out of the box yet with boot.

1. Expand output-dir from current fileset to an actual absolute path (fair enough)
2. Edit the main compiled js file, which is using :none so bootstrapping via other files, to have "base/" prefix in paths (similar issue to asset-path)

During debugging I also made sure that the generated script file and all required files were under a common directory (which I'm not sure was necessary). If it is necessary, this will also take further work for use with boot.

Complaints:

1. Introducing selmer and code generation seems unnecessary. Can you not use env variables or cli arguments to pass the parameters? For example boot-cljs-test would not want to create even temporary files outside the fileset, especially ones linked to the lifecycle of particular filesets.
2. No output whatsoever is shown for broken runners (eg. in this case cannot load all files). Also despite manually running of the command terminating, via the runner it seems to hang indefinitely.
3. Test results coming via console.log is not very readable the way it is broken up and presented by Karma. Here's an example:

Safari 8.0.6 (Mac OS X 10.10.3) LOG: '
Testing rest.app-test'

Safari 8.0.6 (Mac OS X 10.10.3) LOG: '
FAIL in (test-arithmetic) (cljs$test$run_block@http:364:17)'

Safari 8.0.6 (Mac OS X 10.10.3) LOG: 'Something foul is a float.'

Safari 8.0.6 (Mac OS X 10.10.3) LOG: 'expected: (= (+ 0.1 0.2) 0.3)'

Safari 8.0.6 (Mac OS X 10.10.3) LOG: '  actual: (not (= 0.30000000000000004 0.3))'

The good news seen above is that Safari seems to work as well as any of the other runners :smile:

On the downside, having been too proud (slash, it's late where I am and going to sleep now) to launch Leiningen, I haven't been able to run this end-to-end yet, only run the runner command directly from the CLI after fixing the files manually.

[bensu]

Great feedback! Thanks!

I'll take your concerns one by one and make individual issues to address them:

1. output-dir: will be handled sooner than later (#15), I recently noticed that the cljsbuild extract option introduces some problems.
2. I don't know why that wouldn't work, did you have to specify asset-path? Can you elaborate?
3. I don't control the karma runner. The code generation is for a config file that is written in Js. Since I don't choose how the code code is executed, I don't know how to pass/pickup cli arguments. Environment variables are an option, since they can be retrieved from the config file but they are a lot of indirection, from an options map -> environment variables -> to conditionals inside the js. On the generated file, I'll try to make it an absolute path from temp and see if that works. Remember that all the other runners are also generating a temp file, it is just not done by templating (#22).
4. Yes, it is very annoying (on the docs under Troubleshooting Karma). Karma doesn't fail but stops and asks you if everything is ok. If it doesn't terminate, I don't regain control form Clojure and don't have stdout to print. I'll need to make changes to how sh is used in order to print what happens in Karma as it is happening. (#23)
5. Agreed. That is another day of hacking to get cljs.test/report to do the right thing and play along with Karma (#24)

Thanks for doing this!

[bensu]

Hi @crisptrutski!

I recently addressed the biggest problem around relative paths (#15) that might have prevented boot-cljs-test from using doo.core/run-script. Also, I removed Selmer and moved the generated files into /tmp (#22). Can you try running boot-cljs-test with karma again in the not released [0.1.6-SNAPSHOT]?

Since browser testing is now properly working for the library and the plugin, I'll be closing this issue. We can handle whatever happens next in new issues.