Support Mutation testing use case

[nicojs]

🙋‍♂️ Hi, we're from StrykerJS, the mutation testing framework for JavaScript and friends. We plan to add support for the Web Test Runner (WRT) in one of our next releases.

To do that, we would love to use a (public) programmatic API. Would you be willing to accept PRs to document the current test runner node API? And also PRs that extend the API with some new functionality, namely:

• A way to load the user config, slightly change it (add a plugin), and pass that to the TestRunner instance
  • Might be possible by opening up that API.
• A way to filter tests and test files (sessions?)
  • We might need to interact with mocha? I don't think this is currently possible.
• A way to set a global variable
  • Probably possible by adding a virtual file using a plugin?
• A way to communicate mutation coverage (an object on the global scope) from the browser to the main process
  • Might be possible with a plugin?
• Being able to scan what tests will be scheduled to run before running them. So, sessions need to be initialized and accessible before a test run is executed.
  • Not possible at the moment I think
• Running a hook before each test is executed.
  • Might be possible by adding a beforeEach hook in a virtual file.

We're wondering if you would be open to this. Do you want us to create an issue for each change? Or can we create PRs per change? Any help would be really appreciated 😅

See also 4489

[Westbrook]

An issue for each change would be a great start. It sounds like you’ve already investigated the project a good bit, so some pointer as to where you’d think changes would be necessary would be a good inclusion there. No grantees all changes can land, but isolating them for further discussion will certainly smooth the way.