lukeed / uvu

uvu is an extremely fast and lightweight test runner for Node.js and the browser
MIT License
2.98k stars 99 forks source link

Timeout control #33

Open Hebilicious opened 4 years ago

Hebilicious commented 4 years ago

With jest it's easy to control the timeout iejest.setTimeout(5000). Any chance you could implement such a thing ? Really enjoy uvu so far !

lukeed commented 4 years ago

Hey, I'm glad you're enjoying it :)

I don't plan to, at least in the same way described, because uvu shouldn't modify your code. If something takes too long, an error should be thrown because it's obviously not what you want to happen.

A timeout assertion could be added, but I'd have to come up with a different name though so that assert.timeout and assert.not.timeout makes sense -- perhaps that already does?

// Single function check
assert.not.timeout(() => something(), 2000);

// Put everything inside, easier?
assert.not.timeout(async () => {
  let foo = await something();
  assert.equal(foo, bar, 'inner assertion');
}, 2000); 

// Could do this too 
let foo;
assert.not.timeout(() => foo = something(), 2000);
assert.equal(foo, bar, 'inner assertion');

If the parameters were swapped ((ms, func)) then it allows for you to bind a saved function with your own value:

const timer = assert.not.timeout.bind(null, 5000); // 5s

//...

timer(() => something());
timer() => something(), 'custom message');
Hebilicious commented 4 years ago

I'm not familiar with the implementation details of jest.setTimeout. The API you propose is quite nice. I was thinking of something that can be set globally However, I'm under the impression that it's just a matter of having it configured and using a timeout logic in the library ? I'm not too familiar with your codebase, but I assume you'd throw an error if a test takes longer than the timeout and then write FAIL, probably around here : https://github.com/lukeed/uvu/blob/4e2dce42dd3b741d690a8fb374793fee295508c8/src/index.js#L89 if you see what I mean ?

I was thinking of a CLI flag like this :

uvu -r esm -r esbuild-register tests --timeout 60000 #a test will fail if it takes longer than 60s

Happy to make a PR as well, just not sure it's the right approach.

lukeed commented 4 years ago

For sure, that's definitely possible but it does incur overhead/cost to all other tests and suites, even if they don't care to have it.

Personally it makes more sense as an assertion method to opt into. It reads a lot like assert.(not.)throws in my mind, which you could also do globally but you only care about it at particular points

tinchoz49 commented 3 years ago

Hey @Hebilicious I don't know if it helps but in our tests we do something like this:

const { test } = require('uvu')
const assert = require('uvu/assert')

const timeout = (handler, ms) => {
  return (context) => {
    let timer
    return Promise.race([
      handler(context),
      new Promise((resolve, reject) => {
        timer = setTimeout(() => reject(new Error('timeout')), ms)
      })
    ]).finally(() => {
      clearTimeout(timer)
    })
  }
}

test('should fail', timeout(async () => {
  // a promise that never ends
  await new Promise(() => {})
  assert.ok()
}, 5000))

test.run()

This is why I really like uvu, it's just JavaScript. You can do a timeout feature in so many ways.