vitest-dev / vitest

Next generation testing framework powered by Vite.
https://vitest.dev
MIT License
12.25k stars 1.08k forks source link

Automatic expect tracking for concurrent tests #5665

Open XantreDev opened 2 months ago

XantreDev commented 2 months ago

Automatic expect tracking for concurrent tests

It's cumbersome and error prone to accept expect for each concurrent test Automatic tracking is also allows to make tests faster by using .concurrent wrapper, without code modifications

import { describe, it } from 'vitest'

// All tests within this suite will be run in parallel
describe.concurrent('suite', () => {
  it('concurrent test 1', async ({ expect }) => { /* ... */ })
  it('concurrent test 2', async ({ expect }) => { /* ... */ })
  it.concurrent('concurrent test 3', async ({ expect }) => { /* ... */ })
})

Suggested solution

We can add opt in option to use AsyncLocalStorage for node.

import { AsyncLocalStorage } from 'node:async_hooks';

const expectStorage = new AsyncLocalStorage();

const getTestExpect = () => expectStorage.getStore()

const expect = (...args: Parameters<typeof originalExpect>) => {
  const localExpect = getTestExpect()
  if (localExpect) {
    return localExpect(...args)
  }
  return originalExpect(...args)
}
const it = (testFunction: Parameters<typeof originalIt>) => {
    originalIt((...args) => {
      const {expect} = args[0]

      expectStorage(expect, testFunction, ...args)
    })
}

Alternative

Zone.js can be also used for async context tracking, since it monkeypatches a lot of stuff - it less reliable

Additional context

No response

Validations

sheremet-va commented 2 months ago

We cannot use AsyncLocalStorage in core since it's a Node.js API. Vitest runner also runs in the browser and should have the same API surface. The best we can do is wait until the context API is standardized.

As a workaround, you can wrap it functions yourself. You can also extend Vitest runner with runner option or use a custom task function.

sheremet-va commented 2 months ago

Another thing I wanted to mention is that using { expect } from a concurrent test is required only and exclusively for snapshot testing and expect.assertions methods.

Normally, expect doesn't need to know the current test - it just throws an error based on the input.

XantreDev commented 2 months ago

So in most cases we can just use global expect. Interesting I will try to explore how to implement it in userland