vitest-dev / vitest

Next generation testing framework powered by Vite.
https://vitest.dev
MIT License
12.68k stars 1.14k forks source link

A Node API to get testing results from the Vitest instance #4257

Open Jinjiang opened 11 months ago

Jinjiang commented 11 months ago

Clear and concise description of the problem

I'm trying to integrate Vitest into my own project via Node APIs. Basically, we need to filter testing files from somewhere in the file system, pass them into Vitest, and display the testing results. The whole process has to be connected and finished as Node APIs.

Now I understand we already have some Node APIs to create or start a Vitest instance. However, there is no way to get testing results from the Vitest instance.

I made a reproduction to walk around by a inline reporter, however, it's not easy to use. I feel it would be great if we could have a Node API on Vitest instance to get the testing results.

https://github.com/Jinjiang/reproductions/tree/vitest-node-apis-20231006

Suggested solution

Some new API on Vitest instances like:

const results = await vitest.getResults()

https://github.com/Jinjiang/reproductions/blob/vitest-node-apis-20231006/node-apis/run-test.mjs#L49

Alternative

Just for ref: some similar APIs exist in Jest community https://medium.com/web-developers-path/how-to-run-jest-programmatically-in-node-js-jest-javascript-api-492a8bc250de

const result = await runCLI(jestConfig as any, [projectRootPath]);

Additional context

No response

Validations

sheremet-va commented 11 months ago

You can already do vitest.state.getFiles() which is equivalent to what you get in onFinished. vitest.state.getUnhandledErrors() will return your errors.

Jinjiang commented 11 months ago

@sheremet-va Oh glad to know that. Thanks! Btw, I feel it deserves a mention on the docs.🙂

Jinjiang commented 11 months ago

A further question: is there any similar API for somehow getting changes or results lively on the watch mode? Like:

vitest.onChange((files, errors) => {})

Thanks.

sheremet-va commented 11 months ago

Thanks.

Only inside reporters. onTaskUpdate is called when the state of a task is changed. Vitest sends this event every ~80ms, so it can contain several tasks if they are executed too fast:

onTaskUpdate(packs) {
  packs.forEach(([taskId, taskResult, taskMeta]) => {
    const task = vitest.state.idMap.get(taskId)
  })
}