Document how to deal with cross-origin restrictions

[jamesshore]

If you use Quixote with a test page that's served from a different origin, Quixote won't be able to access the test frame's elements and will fail. This is due to browsers' same-origin policy.

This is an issue for dynamically-generated pages. What are some options for resolving it?

[dwoldrich]

Perhaps expecting a test page is the wrong way of developing with Quixote. Funneling developers down the Karma-Mocha/Jasmine/Qunit testing route enforces a same-origin, no? I should think other assertion libraries would suffer similarly to Quixote if they were required to access cross-origin content in the iframe. Would it make sense to require quixote.createFrame's options.src (if defined) to be required to be a root-relative URI such that the content must be served via Karma's files?

Are there specific examples are you thinking that would necessitate cross-origin testing?

[jamesshore]

I agree that you'd typically want to use a root-relative URI. But that only works for static pages. If you want to test against a localhost server, it would be running on a different port than Karma, which runs into same-origin restrictions.

Someone could argue that Quixote is for unit testing CSS, and hitting a server--even a localhost server--is actually integration testing. It's a valid argument. But the localhost server use case is so common that it seems like we should support it. Plus, I'd like Quixote to be an assertion library you can use for any kind of testing you like, even if it's designed for unit testing first and foremost.

[dwoldrich]

Your comment has focused my thoughts here. I believe if you want to unit test your CSS and HTML, you would use Quixote. And, if you wanted to test your CSS and HTML on the live site, you'd be better off using something like Selenium because of its web agents that hook the host browser and sidestep the same-origin restrictions. A Java Quixote assertion library that leverages the Selenium WebDriver API for accessing the DOM might be the ticket for CSS integration tests against "real" pages.

That said, recommending Quixote only for unit testing CSS on static HTML+JS pages does not mean it has lost its utility. First, most greenfield projects have their CSS assets well-in-hand now. LESS or SASS do all the bundling for them, and the CSS build is typically automated. So it is very plausible that your users will be able to load the precise mix of CSS rules that would live on their production site into a Quixote test page.

Second, there does seem to be a trend towards developing parts of the page using component architectures. (WebComponents/Polymer, ReactJS) I think component development could be a sweet spot for Quixote since the dynamism and content templates live in client JS.

I know what you're getting at: that a thorough test of CSS requires a target browser loading the exact mix of CSS assets, generated/production-like HTML, and with all the client-side JavaScript executing. Selenium is a better fit for rigging up that test.

I honestly don't think we can know what effect a thoroughly unit-tested CSS would have on your typical big web development project/process. Most/all of the uncertainty and flakiness of large scale CSS development might be eliminated with unit testing.

[jamesshore]

You're right that Selenium is a better fit for whole-system integration testing. I'm not so convinced that Quixote doesn't have a role to play there, though. I haven't tried it yet, but I think it's possible you could load the Quixote assertion library with Selenium and get the best of both worlds. Like this:

• Karma + Quixote: CSS unit testing
• Selenium + Quixote: CSS integration testing

[laurentbourgaultroy]

I see 2 possible options to allow cross origin assertion.

1. proxy the dynamic web page using karma

In the karma configuration file (see the "proxies" section), it is possible to proxy content coming from a different server into the karma server. The browser see it as coming from localhost:9876, or whatever address karma run on.

It would be possible using this to make a dynamic web page accessible to quixote without security restriction : the browser would believe the HTML come from localhost:9876 . Relative URL for the JavaScript or CSS should work without any problems and would be considered as coming from the same origin. In the case of scripts or style with absolute URL, though, there may be some security restriction. It may not be possible to read the CSS rules for example (like what happened in the chapters on fonts in WWP)

Of course, this approach require that the test are running in karma (or to configure a proxy manually).

2. Use the postMessage API for cross origin communication

Another possible approach would be to use the window.postMessage API. It was explicitly designed to allow cross origin communication between different window / frame / tab . According to MDN, IE8 seems to support it.

Going this approach would require the tested dynamic page to embed a quixote script directly. The embedded script would start listening for request to do assertions via window.addEventListener("message");. The unit test could then connect to the frame and do a frame.postMessage with the details of the assertion to execute. The embedded script would execute the request and send the assertion result through event.source.postMessage.

Going that route would have the following consequence

• Any assertion would have to be asynchronous (window.postMessage is non-blocking). It would make testing slightly harder (though with promises it wouldn't be that bad I believe)
• It would require a modification to the tested web page. The quixote script would have to be removed when built for production (maybe with a debug flag, or by removing it from the template, or something of the sort). That make quixote a lot more intrusive.
• Forgetting to remove quixote from the dynamic may be a security risk if someone forget to remove it in production : it give attackers an easy way to manipulate the web page without much restriction.

For that last point, it may be wise to restrict an embedded quixote to only listen to message if the window.location is "localhost". It would only allow unrestricted access when a flag is set. Of course, that flag would be named something scary like "disableAllQuixoteSecurity".

---

Those are the two way I could see cross origin testing work

[dwoldrich]

@laurentbourgaultroy I like the idea of configuring proxies in Karma, etc, to overcome cross-site scripting restrictions. It doesn't address the integration testing in a lot of cases though because AJAX calls the page makes may try to go to the original domain, at which point we fall back into the cross-site restrictions problem.

@jamesshore For integration testing against the prod/pre-prod site via Selenium, also consider building Quixote-esque assertions extensions into something awesome like NightwatchJS. (Which is a fancy NodeJS facade to Selenium WebDriver.) http://nightwatchjs.org/

I don't think Quixote, as it is written, would work in Selenium because the Selenium tests do not run on the browser. Selenium agents run on the browser and the tests written in your favorite language access the agents via the Web Driver.

[jamesshore]

@dwoldrich Selenium WebDriver has the ability to insert JS into the page under test, that's what I meant.

[jamesshore]

@laurentbourgaultroy Using a proxy seems like the right answer here. I've added a to-do to document it.

[jamesshore]

I got a question by email that forced me to explain this a bit better. Putting it here for future reference:

You can use Karma’s proxying capability to get around the cross-origin issues. Add these lines to your karma.conf.js:

urlRoot: '/karma/',
proxies: {
   '/': 'http://localhost:5000/'    // replace with URL of test server
},

After you run the Karma server, you’ll capture browsers by pointing them to http://localhost:9876/karma. Then, before running your tests, run the server you want to test (in your case, your Mvc.Net application).

Your Quixote test code will remain the same, except you’ll use the Karma server rather than your application server for your page URLs. For example, in a test of “/foo/bar.html”, you would use:

quixote.createFrame({
  url: “/foo/bar.html”    // instead of http://localhost:5000/foo/bar.html
}, done);

This will work with any language or host and should completely eliminate all cross-server security issues.

Of course, unit tests such as the ones we demonstrate in Let's Code JavaScript's "Recorded Live" chapter 48 sidestep the issue entirely. I’m still learning about the best way to test CSS, but I’m liking the unit tests better than integration tests so far.

[jamesshore]

This issue's been resolved but still needs to be documented. I'm going to update the title and labels to reflect that.

[jamesshore]

Documented in the v0.10.0 release.