Organize all tests?

[kainino0x]

I've been wanting to consider organizing the CTS to match the table of contents so it is easier to match between the TOC, spec, and CTS. (At this time, the spec also still needs updating to match the TOC. The TOC may also change.) https://github.com/gpuweb/gpuweb/wiki/Table-of-Contents

• Match the TOC
• Maybe it makes sense to merge the operation and validation trees and use a test suffix to indicate the difference (e.g. replace .spec.ts with .op.ts and .val.ts). That will resolve the confusing divergence between those two trees.
• Also consider renaming the webgpu suite, it's redundant, maybe "conformance", "cts", "c"?

It's also possible we shouldn't do this - it'll wreak havoc on test expectation files in the browsers, and the current situation is not that terrible. We should look into Bikeshed's WPT integration and see if it's useful/sufficient to use it to point sections of the spec at tests. (This may be too hard to maintain to be worthwhile though.)

[dj2]

The WPT support in bikeshed requires the tests to be in WPT (the paths are relative, not absolute). So, unless we moved into WPT, or updated bikeshed to allow a different WPT root (at which point is it really WPT?) I don't think it would work well.

The <div assert> change just landed in Bikeshed. So, you could in theory do what we're hoping to do for WGSL which is to markup the spec with asserts and then link those assert IDs from the tests. (I'm not sure how long it takes bikeshed features to roll out to things like the online spec builder or the w3c github action infrastructure though)

[kainino0x]

Good point. We had originally planned to export these tests into WPT (or move them there if it gets TypeScript support or we can import babel or something). But at this point it's looking less likely, as most of the browsers aren't even running them under the WPT harness (testharness.js).

It would be kind of nice if the sources were in WPT for the auto-import/auto-export from browser repos, but I don't see that being possible anytime soon.

[kainino0x]

I think Bikeshed changes usually get rolled out within a week or two (then we also need to update the pinned revision in the PR preview step).