search

gpuweb / cts

WebGPU Conformance Test Suite
https://gpuweb.github.io/cts/
BSD 3-Clause "New" or "Revised" License
127 stars 77 forks source link

Running CTS on React Native #3892

Open wcandillon opened 1 month ago

wcandillon commented 1 month ago

We have built a WebGPU binding for React Native based on Dawn and we also have a small test suite for it which we run against Chrome as well for reference.

We are now looking to run the CTS as well. I tried to bundle this module but been struggling in a few places.

So I wanted to try something simpler manually to get a sense on how things work. I was able to load a TestGroup and it contains a few tests. I couldn't find which APIs I need to use just to execute this single test. Could you give me some pointers?

I will still try to see if I can find a way to package CTS in a way that would fit React Native but if there is a minimum set of API I could use directly where I could build my own test loader, that would be great.

greggman commented 1 month ago

@kainino0x probably knows best but might be out for a few weeks.

I have zero experience with React Native.

In any case, there are multiple ways to run the tests. One way is via standalone.ts. Another is via cmdline.ts. You could look at both of those to see how to write your own runner.

As for what features, off the top of my head, most of the tests don't need more than JavaScript and WebGPU but there are tests, especially in the web_platform folder, that test interaction with WebGL, Canvas2D, ImageBitmap, ImageData, HTMLImageElement, HTMLVideoElement, HTMLCanvasElement, VideoFrame, OffscreenCanvas, requestAnimationFrame and I'm sure a few others

Also, maybe take a look at how dawn.node runs the tests

https://dawn.googlesource.com/dawn/+/refs/heads/main/src/dawn/node/README.md

You can see how it calls into the command line version here to get a list of tests

https://dawn.googlesource.com/dawn/+/refs/heads/main/tools/src/cmd/run-cts/common/cts.go

I didn't dig to see how it runs them.

IIRC dawn.node fails or skips tests under web_platform since I think none of those are implemented in node.

kainino0x commented 2 weeks ago

I think Gregg covered everything! I also do not know anything about React Native.

If React Native's runtime is Node or something similar, then looking at cmdline.ts is probably the best place to start. That has already been used to run WebGPU tests in both Dawn's Node bindings and Deno's bindings for WebGPU.

You should be able to run the whole suite; all tests (regardless of subdirectory) that have dependencies on other APIs, like DOM and browser APIs, should automatically skip themselves when those aren't available.

Would love to hear updates on your progress! Feel free to contribute changes to this repo if needed (but they might not be, since it already works in Node/Deno).

wcandillon commented 2 weeks ago

@kainino0x Thank you for your note. It will be quite a bit of work for us as we will need to do a custom bundle but I'm sure we can get there eventually. Lately any examples we throw at the module work immediately out of the box so we didn't feel this is priority yet but I'm sure this will come up eventually.

Here are demos from the WebGPU samples:

https://github.com/user-attachments/assets/116a41b2-2cf8-49f1-9f16-a5c83637c198

And then we tried to run Three.js on top of it and it kind of worked out of the box:

https://github.com/user-attachments/assets/5b49ef63-0a3c-4679-aeb5-e4b4dddfcc1d

But I definitely want to revisit this topic.

kainino0x commented 2 weeks ago

That's very nice, thanks for sharing!