Async test cases in browser, useful for AMD, RequireJS

[augustl]

Buster currently can't facilitate these types of tests:

require(["my/file"], function () {
    buster.testCase("blah", {
        // ....
    });
});

The difficulty lies in knowing when all the buster.testCase functions have called. body.onload is not enough, due to the async require. We need to know when the test run has "finished" so we can end the session in buster-capture-server and exit the "buster test" process in the terminal, so we can't just wait 10 seconds and hope all the tests have loaded by then..

We need some sort of counter to let buster know before body.onload how many test cases that it expects to load. Suggestion:

buster.asyncTestCase(function (testCase) {
    require(["my/file"], function () {
        testCase("My test case", {
            // ....
        });
    });
});

With this model, buster knows at body.onload that there is one async test case that will be loaded in the future. It can wait for the passed "testCase" (which is just a wrapped buster.testCase) function to be called and know that if it isn't called it can timeout, and so on.

[geddski]

@johlrogge I'd like to try out your extension. Let me know when the docs are ready.

[cjohansen]

All right, we're getting there. We're aiming for a new release next week. For that release I'd like the AMD extension to ship as an "official" one. I just put in some work to update @johlrogge's code to work with the significantly revamped APIs: https://github.com/busterjs/buster-amd/blob/master/lib/buster-amd.js

[briancavalier]

@cjohansen That's great news! I'll tag @unscriptable too, cuz I know he'll be interested to see the progress.

[johlrogge]

@cjohansen yay! Awesome to see my baby become revamped and official!

@geddesign I'll have a look at the docs during the weekend. I have been pretty swamped so far, there are bits and pieces in this thread

[cjohansen]

Question: do all the loaders provide the require function, or would it make sense to somehow make that call configurable?

[johlrogge]

I have only used requirejs so I don't know. How about postponing the configurability until need arises? I got curious though about how a plugin can have custom configurations. Is that new after the latest changes to the API's? Any pointers to how it works?

[cjohansen]

Yes, it's new :) I've started some very rough docs here: http://busterjs.org/docs/extensions/

You can also see the example in the test: https://github.com/busterjs/buster-amd/blob/master/test/buster-amd-test.js#L87

[briancavalier]

The global require() function isn't actually part of the AMD spec (but global define() is), so imho, the name of the loader function should be configurable. Some loaders have chosen to use the name "require", but some haven't. For example, curl's global loader function is curl().

+1 for configurability :)

[cjohansen]

@briancavalier does that mean that using define(["test1", "test2"], function () {}) would be a viable option that would work for every loader? Or would that require the "bundle" to be required (or 'curled')?

If the define suggestion doesn't work, is it enough to make the name of the loader configurable? Do they have the same interface?

[briancavalier]

Good question. I think that ultimately, you'd still need to call curl/require in order to have the loader execute the bundle define, as you said. I know that curl and requirejs's loader functions have compatible signatures, so at least for those two loaders, configuring the name should be enough. Beyond that, I'm now sure how widely they vary, but my guess is that most will be compatible.

If it ends up needing to be more sophisticated for some reason, maybe allowing a string with replaceable tokens, or even a function to provide loader specific output would be an option.

[johlrogge]

Perhaps configure a "runner" function:

runner: function(mod) {require(mod)}

or in case of curl (given that the syntax is compatible): runner: function(mod) {curl(mod)}

if require is most common the first version could be default ?

[briancavalier]

@johlrogge I like that idea a lot. So maybe the buster amd plugin generates a define bundle, as @cjohansen said, and then uses the runner function to "require" it, causing it to execute?

[johlrogge]

@briancavalier Yes, you understood me correctly inspite of me leaving out the reference to @cjohansen's define-idea.

[geddski]

@cjohansen curious if your run() implementation is still in, after the rewrite and this extension. I think it could still be useful.

[cjohansen]

@geddesign Sure, it's still there.

[cjohansen]

@briancavalier @johlrogge I'm not following anymore :) Could you show me an example config of how that would look?

[johlrogge]

config["Browser tests"] = { rootPath: "../", sources: ["src/*/.js"] tests: ["test/*/.js"], extensions: ["buster-amd"] bootstrap: function(mod) {curl(mod)} };

If scoping of curl is an issue the maybe it can be solved with an eval:

config["Browser tests"] = { rootPath: "../", sources: ["src/*/.js"] tests: ["test/*/.js"], extensions: ["buster-amd"] bootstrap: "function(mod) {curl(mod)}" };

I choose the name bootstrap rather than runnner since having runner mean two different things would be rather confusing and imo bootstrap better reflects that it happens before running

[johlrogge]

Just remembered, It is pretty vital to be able to specify where the loaderModule is created (so that one can make sure that it is created where the "production" main module is located. Otherwise there are issues with paths (if a module loads other modules they are loaded relative to the loaderModule and if the loader module is not at the same level as the "main" module this will fail)

Short version: it must be possible to generate the loader module in the same location as the production "main" module

[cjohansen]

Ah, so the main loader should live at the root of the resources then? Currently it gets put in /buster/something.js which I guess won't work.

[briancavalier]

I think it depends on a few things. First, there's the global loader function, i.e. curl() and require(). These both load modules based on their baseUrl configuration, which is relative to the url of the HTML page in which their script tag appears. For example, if there's an HTML page at foo.com/stuff/index.html that includes a script tag for curl, and curl's baseUrl is set to "js/", the global curl() will load modules starting at foo.com/stuff/js. So, curl(['my/module']) will attempt fetch foo.com/stuff/js/my/module.js (for now, let's assume there are no other curl path or package configs specified).

Then, there is the local require, which is a part of the AMD spec. It's the require that's magically passed to a module definition:

define('my-module', ['require',...], function(require, ... ) {
    // Use local require here
}

Modules loaded by the local require will be based on:

1. Relative module ids, e.g. './my-helper-module' will be loaded based at the same url as my-module,
2. Absolute module ids, e.g. 'my/other/module' will be loaded from the global baseUrl as usual

I'm not sure how this fits into the current scheme of buster's browser testing, but it seems like the having curl or requirejs be able to correctly determine their own starting location, and then have their baseUrl config point to the "right spot" will be key.

[cjohansen]

Hmm. I think it's kinda given that you have to provide your own loader, and possibly also configuration of that, e.g. through the libs (or testLibs) config options. In this configuration you have access to buster.env.rootPath, which gives you the base path from where the loader is run.

I'll make the loader call configurable, then you guys can start trying this out next week, when the new version is in NPM. Then we can work on other configurability issues once we get a feeling of how it works.

[briancavalier]

Sounds good, can't wait to try it out.

[johlrogge]

Really looking forward to try it out. I feel I dumped something half baked on you @cjohansen but I'll defend myself with that the plugin API did not support custom confifguration options when I made the first version :)

@briancavalier thanks for pointing us to curl, it looks really interesting: I will definitely try it out on my current pet-project

[unscriptable]

Just catching up on this thread now. Looking forward to trying this!

[cjohansen]

@johlrogge Do you have a simple example app that I can use to make sure buster-amd is ready for prime-time? We're about ready to relase.

[johlrogge]

Nothing that I can wrap up quickly and ship. But if the developer scripts are up to date I could update my local buster and see if it works with what I have. I may be able to distill some tests that tests a semplate-loader for knockout if that helps

[johlrogge]

I tried to update my system with buster-dev-tools cloned from github, went so so. It seems like all projects that depend on "when" fail to npm link. Here is the error. Is there some kind of workaround for this issue. Is it only me or is there some problem with those tars? Would it help if I tried to upgrade my npm or is it no use?

Updating projects: ........................ Symlinking dependencies: .................................................................................... npm linking: ............{ name: 'buster-resources', gitUrl: 'git://github.com/busterjs/buster-resources.git', localPath: '/Users/jocke/opt/buster/buster-resources' }

/Users/jocke/opt/buster/buster-dev-tools/functions.js:145 throw err; ^ Error: Command failed: npm ERR! couldn't unpack /var/folders/tD/tDwLTkyLGGWnSfzxJnNduk+++TM/-Tmp-/npm-1329261674076/1329261674076-0.6187632577493787/tmp.tgz to /var/folders/tD/tDwLTkyLGGWnSfzxJnNduk+++TM/-Tmp-/npm-1329261674076/1329261674076-0.6187632577493787 npm ERR! Error: ENOENT, no such file or directory '/var/folders/tD/tDwLTkyLGGWnSfzxJnNduk+++TM/-Tmp-/npm-1329261674076/1329261674076-0.6187632577493787/package/package.json' npm ERR! Report this entire log at: npm ERR! http://github.com/isaacs/npm/issues npm ERR! or email it to: npm ERR! npm-@googlegroups.com npm ERR! npm ERR! System Darwin 9.8.0 npm ERR! command "node" "/usr/local/bin/npm" "link" npm ERR! cwd /Users/jocke/opt/buster/buster-resources npm ERR! node -v v0.6.5 npm ERR! npm -v 1.1.0-alpha-6 npm ERR! path /var/folders/tD/tDwLTkyLGGWnSfzxJnNduk+++TM/-Tmp-/npm-1329261674076/1329261674076-0.6187632577493787/package/package.json npm ERR! code ENOENT npm ERR! message ENOENT, no such file or directory '/var/folders/tD/tDwLTkyLGGWnSfzxJnNduk+++TM/-Tmp-/npm-1329261674076/1329261674076-0.6187632577493787/package/package.json' npm ERR! npm ERR! Additional logging details can be found in: npm ERR! /Users/jocke/opt/buster/buster-resources/npm-debug.log npm not ok

[briancavalier]

What version of when is buster using? Using when from npm is kind of in a state of flux right now, but I wonder if this has to do with when.js v 1.0.x having switched to the name 'cujo-when' in its package.json. If that looks like it's the problem, let me know ASAP, and we can figure out a solution.

[cjohansen]

We're depending on 0.11.1 tar file from GitHub which seemed to have some problems yesterday.

@briancavalier is cujo-when 1.0 pushed to npm? If so I can update the dependency. That would also avoid this becoming a problem when people install buster from npm.

[cjohansen]

To answer my own question: No, doesn't look that way http://search.npmjs.org/#/cujo-when

[neonstalwart]

why not use the github url as the dependency and avoid the npm registry - http://npmjs.org/doc/developers.html#What-is-a-package

[cjohansen]

@neonstalwart That's possible of course, it's what we do now. However, that also makes us depend on both NPM and GitHub at install time. If we can avoid it, I'd prefer to have all the dependencies in NPM...

[briancavalier]

The best option for cujojs right now is to move forward with github urls, rather than npm. At least until npm realizes that a single global namespace is not sustainable long-term (other loaders--all AMD loaders, and even PINF, which is a CJS loader--allow aliases and namespacing)

There are some tricky issues with sharing modules across node and AMD right now, and all of them could be solved if npm was more flexible, especially when dealing with nested dependencies. For example wire.js depends on when.js, both of which have existing users that already have AMD configurations built around the AMD module id 'when'.

We'll do a when.js 1.0.2 release ASAP that changes the package.json name back to 'when' so that installing it via github url gives you a node package named 'when'.

[neonstalwart]

exactly - urls scale better as a namespace :) and @cjohansen if you don't want to depend on 2 things then make everything github urls. it also means there is no latency in publishing - as soon as you push its available.

[cjohansen]

Hehe. Well, @briancavalier I understand. We'll move forward with GitHub tarballs then. @johlrogge: Your problem yesterday was probably (hopefully) temporary, as the 0.11.1 tarballs (which we depend on) are still around.

[briancavalier]

@cjohansen I just pushed when.js 1.0.2

@neonstalwart completely agree about urls being a great choice for package namespaces :) I tweeted about it recently and got a "that's never going to happen" reply ...

[augustl]

We'll do a when.js 1.0.2 release ASAP that changes the package.json name back to 'when' so that installing it via github url gives you a node package named 'when'.

Ah, so that was why npm 404'd.

[briancavalier]

@johlrogge @cjohansen I think I see the problem with the when dependency in buster's package.json. The url has a 'v' in the version number, but I think the tag is actually just '0.11.1'. If you still have problems, try changing the dep url to:

https://github.com/cujojs/when/tarball/0.11.1

And if you update to 1.0.2:

https://github.com/cujojs/when/tarball/1.0.2

[cjohansen]

Ok. 1) I agree, URLs as package identifiers would make a lot more sense. 2) I still quite understand the AMD/NPM problem as I thought both systems used anonymous modules (except for the installation identifier). 3) I'll update our dependencies to the 1.0.2 tarball.

This'll be sorted out tonight (my time, GMT+1). Thanks everyone for chipping in!

[briancavalier]

Thanks everyone for rolling with the punches on this one!

[geddski]

Epic long issue discussion. Looking forward to the end result.

[johlrogge]

@briancavalier Thanks Brian, removing the 'v' in v0.11.1 solved the issue

[johlrogge]

I have managed to install everything but for some reason "require" is not found in the browser. I tried it in chrome and firefox. I managed to capture the generated HTML: http://pastie.org/3390255 and it looks like require is loaded (I have a bundled requirejs and jquery (require-jquery.js)

This is what the runner sais:

Uncaught exception: Uncaught TypeError: Property 'require' of object [object DOMWindow] is not a function Chrome 17.0.963.46 OS X: Uncaught exception: require is not a function Firefox 3.6.16 OS X:

I threw together a quick and dirty example that gives the above results and posted it here: https://github.com/johlrogge/buster-amd/tree/master/demo

Do you guys get the same issue?

[briancavalier]

Is there a script tag somewhere that is pulling in an AMD loader (e.g. curl.js)? If not, that raises the question of how to "load the loader" as they say. Is there a way to have buster server use a custom HTML page for browser tests? If so, maybe putting a script tag (and loader config, most likely) in that custom page might do the trick.

[briancavalier]

Oh, duh, I totally missed that you are pulling in require-jquery, sorry! Maybe somehow require is being invoked (maybe by buster) before the require-jquery.js script is loaded?

[johlrogge]

@briancavalier yes it seems require is being invoked prematurely. I could kill for a stacktrace right now and I don't know my way around firebug or chrome developer tools well enough to get one. I gave up yesterday. I was thinking to modify the generated page and run it locally just to peel away all the faye stuff and hopefully be able to reproduce it in a more controlled environment but... hopefully someone can come up with a better idea before I loose any more hair :)

[augustl]

Breakpoints (in order to get a stack trace) largely depends on #22, I'll see to that getting fixed soon. Then we probably also need some sort of mechanism to "pause" the test run, so that you don't have to be fast and set the breakpoint before the tests suite finishes running.

[unscriptable]

Hm. A stack trace or breakpoint would be lovely here. We can't even tell if this is a call to the global require() or an unintentional call to a "local" require() in a module that may have not been wrapped properly.

On Thu, Feb 16, 2012 at 7:26 AM, August Lilleaas < reply@reply.github.com

wrote:

Breakpoints (in order to get a stack trace) largely depends on #22, I'll see to that getting fixed soon. Then we probably also need some sort of mechanism to "pause" the test run, so that you don't have to be fast and set the breakpoint before the tests suite finishes running.

---

Reply to this email directly or view it on GitHub: https://github.com/busterjs/buster/issues/15#issuecomment-3999559

[cjohansen]

Here's a ghetto breakpoint for ya: alert("Stop! Hammer time!"). Heh. @augustl should probably get crackin' on #22

[johlrogge]

@cjohansen does the example project I put together look correct? Me and @augustl tried to figure out how it should be configured over irc yesterday. Would be a bit silly if it is my example that is wrong... Just doublechecking