timeout issues with concurrent tests - not serials

[ghost]

Hi

I was just playing around and testing things etc, so hard to reproduce this. But here is the stack trace:

    fail fast
fail 1 - concurrent (5002ms)
TimeoutError: operation timed out
    at afterTimeout (xxx\ben\node_modules\bluebird\js\release\timers.js:45:19)
    at Timeout.timeoutTimeout [as _onTimeout] (xxx\ben\node_modules\bluebird\js\release\timers.js:75:13)
    at tryOnTimeout (timers.js:228:11)
    at Timer.listOnTimeout (timers.js:202:5)
pass 2 - serial (4ms)

Total: 2
Failed: 1 (1)
Passed: 1 50%

The test that is not failing is the serial one.

unhandledException
AssertionError: 'fail' === 'stop'
    at Runner.SpyReporter.runner.on.suite2 (xxx\ben\test\fail-fast.js:14:15)
    at emitOne (events.js:96:13)
    at Runner.emit (events.js:188:7)
    at Runner._emitSuiteEnd (xxx\ben\lib\runner.js:67:10)
    at Promise.resolve.catch.then.then.then.catch.then (xxx\ben\lib\suite.js:230:30)
    at tryCatcher (xxx\ben\node_modules\bluebird\js\release\util.js:16:23)
    at Promise._settlePromiseFromHandler (xxx\ben\node_modules\bluebird\js\release\promise.js:504:31)
    at Promise._settlePromise (xxx\ben\node_modules\bluebird\js\release\promise.js:561:18)
    at Promise._settlePromise0 (xxx\ben\node_modules\bluebird\js\release\promise.js:606:10)
    at Promise._settlePromises (xxxnode_modules\bluebird\js\release\promise.js:685:18)
    at Async._drainQueue (xxx\ben\node_modules\bluebird\js\release\async.js:138:16)
    at Async._drainQueues (xxx\ben\node_modules\bluebird\js\release\async.js:148:10)
    at Immediate.Async.drainQueues (xxx\ben\node_modules\bluebird\js\release\async.js:17:14)
    at runCallback (timers.js:570:20)
    at tryOnImmediate (timers.js:550:5)
    at processImmediate [as _immediateCallback] (timers.js:529:5)

[ghost]

After some digging and searching I think part of the issue is related to _emitSuiteEnd

[ben-page]

Will you provide the code that produced this error?

[ghost]

The code was very simple. I wrapped a setTimeout around the test before running it.

setTimeout(function(
... test here!
){ }, 3000);

It doesn't throw this anymore, but if you try to run tests quickly after each other it now says ...add test after suite starts running followed by some errors etc

[ben-page]

The error message is "can't add test after suite starts running" and that's accurate. You can't add tests asynchronously. The runner waits a tick and then all the suites/tests start running. Adding tests asynchronously would be a nightmare. The suite could be completely finished running for you know.

[ben-page]

I don't know what exactly you mean "try to run tests quickly after each other", but it isn't necessary with node-test. It runs all tests concurrently. Meaning they all start and run at the same time. They don't wait for the other tests to complete first. There is absolutely no reason to create the tests asynchronously. If you want to delay a test, put the timeout inside the test.

suite.test(done => {
  setTimeout(() => {
    done();
  }, 3000);
});

[ghost]

I close this. It works okay now,.

[ghost]

The main reason I was trying this was because this is a allowed feature in Mocha. Look at the DELAYED ROOT SUITE section. http://mochajs.org/

[ben-page]

Mocha is not concurrent. Supporting this would add a lot of complexity and I'm not sure how valuable it is.