microsoft / vscode-test

Testing utility for VS Code extensions
MIT License
246 stars 57 forks source link

Running tests with a bundler should be easier #186

Open connor4312 opened 1 year ago

connor4312 commented 1 year ago

Currently our standard test setup is using mocha and globbing for files. This doesn't work well with bundlers like esbuild or webpack. There are hacks to allow star imports in both these bundlers, but ideally we could figure some less-intrusive way to deal with these.

zm-cttae-archive commented 1 year ago

Webpack removes the need to glob for files, but a bundled file that requires test env CLI cannot be used with this package:

Ideally the core repo needs to add CLI argument for an absolute bin path to wrap the script JS.

connor4312 commented 1 year ago

Yea, webpack would, though we would dislike 'hardcoding' to a single bundler. And I myself prefer esbuild for my extensions 🙂 There are solutions there too, of course, but the question is whether we want to maintain a list of instructions/plugins needed for each bundler (which is sure to change over time)

zm-cttae-archive commented 1 year ago

Fwiw there is now bleeding edge BDD test closures shipped with node - Test runner (NodeJS).
It's not on the same version that VS Code ships with unfortunately, but yeah.

remcohaszing commented 1 year ago

I’m trying to understand this problem. Is this correct?

There are integration tests. These tests run test actual behaviour based on the generated bundle that ships with the VSCode extension. These do not require access to the source code. These integration tests are often written in TypeScript, but compiled and run as JavaScript. There’s no need to bundle tests for this.

There are unittests. These test the source code on a lower level. Both the unittests and source code are often written in TypeScript, but compiled and run as JavaScript.

This means tests need a compilation step which is different from the build step.

So the real problem is extension tests can’t run TypeScript code, right?


I imagine the ideal situation would be that the user just needs to write a standard test file, run code-test, and everything just works.

I think this is possible by:

zm-cttae-archive commented 1 year ago

I have a working test suite that scales to Node and Web in a platform-independent manner:

https://github.com/vscode-toolroom/vscode-textmate-languageservice/tree/11223728567c20d66e0966b96290f9e3328f9a4d