Closed connor4312 closed 1 year ago
Personally I am hoping greater use by the team of the native testing UI will lead to the coverage API getting some attention. In particular, until its results get surfaced in the UI there's not a lot of incentive for us extension developers to invest in it. See #123713
I’m a big fan of snapshot testing that Jest supports and think we could benefit greatly from adopting it.
Snapshot tests (aka "expect tests) are tests that capture the string representation of a given object and compare to the expected string. The important aspect of these tests is that those snapshots
toMatchInlineSnapshot()
and the test runner runs the test, get the expected output and updates the test with the expected snapshot. This allows writing tests a lot fasterhttps://github.com/microsoft/vscode/assets/16353531/f5550aed-b47c-4769-85ab-442524f33715
Snapshots also allow for more visual tests because one doesn't need to write those by hand:
test('returns the positions of all functions in the source code', async () => {
const source = outdent`
function add(a, b) {
return a + b;
}
function subtract(a, b) {
return a - b;
}
`;
expect(await jsSourceWithFunPos(source)).toMatchInlineSnapshot(`
"<start-0>function add(a, b) {
return a + b;
}<end-0>
<start-1>function subtract(a, b) {
return a - b;
}<end-1>"
`);
});
I like this blog post describing how snapshot tests can be nice - https://blog.janestreet.com/the-joy-of-expect-tests/
Jane Street is big on snapshot testing and have their own framework. Their way of creating snapshots is a bit different because they don't compare the snapshot against an object but they capture stdout from a test as a snapshot.
Happy to jump on a call to discuss this more and, in general, help with implementation some of the things, if time permits.
Followup items from discussion:
isPeekVisible
context key does not seem to behave correctly 🤔 I finished the big themes I wanted to implement for this issues with the test CLI and its adoption in core. Copying the notice I sent for posterity
This week I've been working on a new way to run extension tests. It includes a configuration-driven command-line runner, as well as an extension that runs extensions tests in the VS Code UI.
:point_right: For people who own built-in extensions: You can now use this for built-in extensions in VS Code! To onboard your extension: (example)
yarn test-extension -l <label>
on the command line. Do this to validate your config is set up correctly!
b. With the ms-vscode.extension-test-runner extension installed, you can run and debug tests using the VS Code UI, as in the image below:point_right: For people who write extensions: pretty similar to the above! You can use the above as guidelines as well as the documentation and an example.