search

vitest-dev / vitest

Next generation testing framework powered by Vite.
https://vitest.dev
MIT License
13.15k stars 1.19k forks source link

feat: attest integration #6963

Open hi-ogawa opened 3 days ago

hi-ogawa commented 3 days ago

Description

Based on https://github.com/vitest-dev/vitest/pull/6852, now I moved all code to core.

summary

Add new assertions:

and a new option to "enable" them (without the option, assertions become no-op and no-error):

Example on stackblitz https://stackblitz.com/edit/vitest-dev-vitest-ywts26?file=test%2Fbasic.test.ts

example code ```js test('inline', () => { expect(squared).toMatchTypeInlineSnapshot(`(n: number) => number`); expect(() => // @ts-expect-error squared('foo') ).toMatchTypeErrorInlineSnapshot( `Argument of type 'string' is not assignable to parameter of type 'number'.` ); expect( () => // @ts-expect-error squared['a'] ).toMatchTypeCompletionInlineSnapshot(` { "a": [ "apply", "arguments", ], } `); }); ```

todo

tbd

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

Tests

Documentation

Changesets

netlify[bot] commented 3 days ago

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
Latest commit 2ee32da6f179d44ec6dd1be242523c0e25071e59
Latest deploy log https://app.netlify.com/sites/vitest-dev/deploys/67459f0632e1c40008947a19
Deploy Preview https://deploy-preview-6963--vitest-dev.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

pkg-pr-new[bot] commented 3 days ago
@vitest/coverage-istanbul

``` npm i https://pkg.pr.new/@vitest/coverage-istanbul@6963 ```

@vitest/expect

``` npm i https://pkg.pr.new/@vitest/expect@6963 ```

@vitest/mocker

``` npm i https://pkg.pr.new/@vitest/mocker@6963 ```

@vitest/browser

``` npm i https://pkg.pr.new/@vitest/browser@6963 ```

@vitest/coverage-v8

``` npm i https://pkg.pr.new/@vitest/coverage-v8@6963 ```

@vitest/pretty-format

``` npm i https://pkg.pr.new/@vitest/pretty-format@6963 ```

@vitest/runner

``` npm i https://pkg.pr.new/@vitest/runner@6963 ```

@vitest/snapshot

``` npm i https://pkg.pr.new/@vitest/snapshot@6963 ```

@vitest/spy

``` npm i https://pkg.pr.new/@vitest/spy@6963 ```

@vitest/ui

``` npm i https://pkg.pr.new/@vitest/ui@6963 ```

@vitest/utils

``` npm i https://pkg.pr.new/@vitest/utils@6963 ```

vite-node

``` npm i https://pkg.pr.new/vite-node@6963 ```

vitest

``` npm i https://pkg.pr.new/vitest@6963 ```

@vitest/web-worker

``` npm i https://pkg.pr.new/@vitest/web-worker@6963 ```

@vitest/ws-client

``` npm i https://pkg.pr.new/@vitest/ws-client@6963 ```

commit: 7d0bfd5

mrazauskas commented 2 days ago

Perhaps it would be useful to add more complex examples? I tried to play with a recursive type:

import { expect, test } from 'vitest';

type JsonValue = string | number | boolean | JsonObject | Array<JsonValue>;

interface JsonObject {
  [key: string]: JsonValue;
}

function stringify(input: JsonValue): string {
  return JSON.stringify(input);
};

test('stringify', () => {
  expect(stringify).toMatchTypeInlineSnapshot(`(input: JsonValue) => string`);
});

Do you think this snapshot is good?

I mean, it will fail if name of the argument gets changed, although that’s not really important. But it will pass if type is redefined into type JsonValue = Record<string, unknown>, although this is exactly the change one would like to catch. Did I miss something?

JSX components were mention in #5857, so here is an attempt:

interface ButtonProps {
  text: string;
  type?: "reset" | "submit";
}

function Button({ text, type }: ButtonProps) {
  return <button type={type}>{text}</button>;
}

test('Button', () => {
  expect(Button).toMatchTypeInlineSnapshot(`({ text, type }: ButtonProps) => Element`);
});

Good to see names of props, but if type would be marked required, that will not get caught. Also note that JSX.Element got stringified as Element. Namespace is lost. I have noticed this previously in tests of DefinetelyTyped libraries. For instance, Immutable.Map and the built-in Map both get stringified as Map by dtslint.

hi-ogawa commented 2 days ago

@mrazauskas Hi, first of all, thanks for testing out!

About what's printed in the snapshot, we've talked with arktype/attest author @ssalbdivad. If I remember correctly, that's what we can get from ts server api. Normally it should match with what you see when hovering on IDE, but vscode can somehow show namespace without getting stripped.

We're not familiar with this area, so mostly we'll need to delegate to what attest supports for now even if we integrate some features as Vitest builtin. I guess this feature will start as experimental and see if the representation makes sense to users in general.

mrazauskas commented 2 days ago

We're not familiar with this area

Fair enough. I mean.. Are you about to ship this feature knowing that you are not familiar with it?

hi-ogawa commented 2 days ago

In terms of API, I'm mostly only using writeAssertionData from @ark/attest, which analyzes xxx inside expect(xxx) and then output .attest/assertions/typescript.json file.

This in turn uses ts.typeChecker.typeToString API from typescript. https://github.com/arktypeio/arktype/blob/649f70eb2ec48b5203099469c6b59d0b5fe29260/ark/attest/cache/ts.ts#L208-L230 If the limitation is typescript API level, then Vitest cannot change anything for the snapshot though :thinking: