typecheck settings is not respected

Eliav2 commented 4 months ago

Describe the bug

tmp

in the gif you can see:

running npx vitest correctly check typing.
running the test from the IDE extension does not catch type error, even though this file only checks for typing only
this extension should respect filed *.test-d.ts for type tests only tests.

Reproduction

first of all, awesome extension!

to reproduce:

git clone https://github.com/Eliav2/express-typed
// or, checkout exact commit (in case the repo would be changed)
git clone --bare https://github.com/Eliav2/express-typed.git
cd express-typed.git
git checkout 6aec67bfe1c849edfac0a85d41a1df0faa60cb0d

now run npx vitest at root directory, or run tests from the IDE.

OR small example

from vitest docs - Play Online

This extension would not correctly catch type.test-d.ts type errors.

Output

Version

v0.8.6

Validations

[X] Check that you are using the latest version of the extension
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

sheremet-va commented 3 months ago

Vitest shows errors on your example as unhandled source errors. To see the error alongside the test, you need your test file to match the typecheck.include matcher.

But there is indeed a bug where Vitest extension doesn't print any unhandled errors at all.

Eliav2 commented 3 months ago

Type check is enabled and handled correctly in my example by vitest himself. The extension doesn't catch these error because probably it's running with its own configurations and doesn't respect the vitest config used in the workspace

sheremet-va commented 3 months ago

No, the extension doesn't modify your configuration, it just doesn't report them because there is no way to assign them to a test. If you read the output of Vitest in the terminal in your gif, you can see that it says "Source Errors", and not test errors. In the same gif you can see "Type Errors: no errors".

Eliav2 commented 3 months ago

@sheremet-va notice, not "source errors" but "Type errors", which is supported(experimentaly) in vitest. I can see you already linked a pull request with a fix 👍💯

sheremet-va commented 3 months ago

not "source errors" but "Type errors",

From your gif:

which is supported(experimentaly) in vitest.

I know how it works, I wrote it, that's why I am explaining to you that you do not understand how it works. You are not supposed to mix runtime tests and typecheck tests. Vitest reports "source errors" because it doesn't treat your runtime test as typecheck test, but as source code.

The fix doesn't change how tests are reported (it will be a green arrow), you need to update your test yourself. The fix only shows the "Source Errors" (see image) in a test result window.

Eliav2 commented 3 months ago

As long as your pr would fix expectTypeOf and assertType assertions on *.test-d.ts files in the vscode extension we're on the same page

vitest-dev / vscode