This PR adds a GitHub Actions workflow that pulls the latest tests from the WebAssembly/spec repo on a regular schedule (or when run manually) and creates a PR with any new changes.
This mimics the existing process for updating WebIDL tests, which is low risk and low friction. Creating a PR that needs to be reviewed removes the risk of the workflow messing up the wpt repo at the cost of needing manual review of the changes (which is mostly rubber-stamping as in the WebIDL case). I also considered a push model from the spec repo, which could reduce CPU costs and achieve lower latency to propagate changes, but with a substantially more complex solution.
My WPT repo fork has some successful runs that you can look at:
A testing PR to verify that the action handles additions, removals and modifications.
A test run to ensure the scheduled action succeeds even when no changes exist in WebAssembly/spec
This should match the MVP that the investigation is pursuing and could subsequently be expanded to cover tentative tests from WebAssembly/testsuite or elsewhere. This should let the investigation get to a 67% score in the Interop dashboard.
The Interop 2024 WebAssembly Testing Investigation intends to get WebAssembly core tests running on WPT.fyi (alongside the existing JS API and Web API tests).
This PR adds a GitHub Actions workflow that pulls the latest tests from the WebAssembly/spec repo on a regular schedule (or when run manually) and creates a PR with any new changes.
This mimics the existing process for updating WebIDL tests, which is low risk and low friction. Creating a PR that needs to be reviewed removes the risk of the workflow messing up the wpt repo at the cost of needing manual review of the changes (which is mostly rubber-stamping as in the WebIDL case). I also considered a push model from the spec repo, which could reduce CPU costs and achieve lower latency to propagate changes, but with a substantially more complex solution.
My WPT repo fork has some successful runs that you can look at:
This should match the MVP that the investigation is pursuing and could subsequently be expanded to cover tentative tests from WebAssembly/testsuite or elsewhere. This should let the investigation get to a 67% score in the Interop dashboard.
/CC @jgraham, @gsnedders