karma-runner / karma

Spectacular Test Runner for JavaScript
http://karma-runner.github.io
MIT License
11.95k stars 1.71k forks source link

I'd like some help with karma-domino-launcher #3251

Open CaerusKaru opened 5 years ago

CaerusKaru commented 5 years ago

Expected behaviour

We should be able to run Karma on Node without relying on all of the browser APIs so that we can create a launcher for NodeJS

Actual behaviour

The karma.js script relies on many browser APIs and doesn't work easily with a NodeJS launcher (even if that launcher executes the karma.js script in a VM).

See example repo here: https://github.com/CaerusKaru/karma-domino-launcher

Basically unless you're using JSDOM, you can't make a launcher on Node without a browser.

johnjbarton commented 5 years ago

I gather you are having some sort of problem with karma.js, but which one, ./context/karma.js, ./static/karma.js ./lib/middleware/karma.js, or ./client/karma.js?

Three of these, are designed to run in only in the browser: client: establishes the websocket to the server, context: runs the test framework static: a built version

lib/middleware/karma.js is a server part and only runs in nodejs.

I don't think any of these are related to implementing a launcher (which is karma-runner terminology for a plugin that launches and controls a browser running tests).

What is your goal?

CaerusKaru commented 5 years ago

Thanks for the quick reply! Our goal is to have Karma work as a test runner for Angular SSR tests, where Domino is the only thing even close to a browser. Ideally that would mean creating a launcher that creates just an execution context for the tests in Node.

johnjbarton commented 5 years ago

Does the rendering service run under Domino? I assume the service must have some DOM and DOM context apis?

The client and context don't use a lot of API, but they live in html pages. And the client uses websockets (of course expecting to be the browser-side websockets).

You might have to use a customClientFile, http://karma-runner.github.io/3.0/config/configuration-file.html

CaerusKaru commented 5 years ago

Yeah that might be the best bet.

Some more context:

CaerusKaru commented 5 years ago

As an aside though, there doesn't look like there's a reason this shouldn't be working in principle. Domino has all of the globals needed for execution, and the script actually makes it pretty far. It looks like there's just a missing piece in getting the context working.

To answer your original question, I was talking originally about the client karma.js, which is the one initially loaded in the browser and fetched by Domino. The following seems to work:

  1. VM runs socket.io and sets up the globals for sockets
  2. VM runs client karma.js and connects to socket io
  3. A new script is added to the DOM which sets up the socket to upgrade to websockets
  4. The VM executes this new script
  5. Nothing yet

I've updated the repo with the new logic to show the progression. But I don't think it should require a custom client file to get this to work. It's probably only missing one or two elements.

johnjbarton commented 5 years ago

From the domino page:

Domino doesn't execute scripts nor does it download external resources.

So I see that you are booting karma client side logic into the domino VM programmatically.

Have you considered using jasmine directly? In this approach you would boot the VM as you have done, then inject your code-under-test, jasmine, your jasmine-runner code, and finally your test code. your jasmine-runner code would define a reporter to store results in a global. Then your boot code would poll the state for the result store to record the test results. You may also be able to share a promise into the sandbox to avoid polling (fun to try but not essential).

Most of karma-runner's value-add is in controlling and communicating with the browser process. But you don't really have a browser or a process.

CaerusKaru commented 5 years ago

The thing about Jasmine is that if we wanted to do SSR testing, we could do it with Jasmine directly. Ideally we would use Karma to give us the independence of testing framework and watch mode, both things we couldn't get with Jasmine.