How do you deal with asyncronous errors in tests?

[fresheneesz]

If you have an asynchronous test, you can use deferred.resolve() to indicate that a test run is complete. But how do you indicate that the test run had errors? There are a few problems with the asynchronous mechanism currently:

A. If a synchronous error happens, jsperf derps on it and just hangs forever waiting for deferred to be resolved. Synchronous errors should be handled the same way as in synchronous tests - test failure. B. The FAQ (being the documentation linked to from the site) doesn't indicate what kind of object deferred is - only that it can be resolved? I tested if it could be rejected (like a jquery deferrred), it either can't, or rejecting it has no effect. C. If a synchronous error is caught and then the deferred is resolved, and then the error is thrown - the test pretends everything is fine! It just ignores the error, and in my case, that means that the test looks not only like it was successful, but that its the fastest running test.

The error mechanism for asynchronous tests is just broken : / . I'd recommend giving users a way to reject the deferred with an error, and appropriately mark the test as having errored. Also, please handles synchronous errors correctly for async tests.

[jdalton]

Ya the deferred object isn't a Promise, just a plain object. jsPerf isn't to validate your code. If you provide it garbage it will execute garbage.

[fresheneesz]

@jdalton Ok, guy. Can you like, try to have a conversation before you close tickets? You're a little trigger happy here.

I have a case where IE has an error, other browsers don't. IE simply can't use one of the libraries being tested in there, but the test should be in there for browsers in which that library works. The ideal case would be a message on that test saying "IE unsupported". Do you get what I'm talking about?

[jdalton]