Implement WebGPU in Servo

[zakorgy]

One of the goals of the University of Szeged team in this year is to implement the WebGPU API (https://gpuweb.github.io/gpuweb/) in Servo. This includes the following tasks:

• [x] Set up a background thread where we call the WebGPU functions (WebGPU thread). Here we would use wgpu-core crate.
• [x] Establish the asynchronous communication between this background thread and the WebGPU DOM objects in the script thread.
• [ ] Implement the WebGPU WebIDL bindings in script\dom, including but not limited to:
  • [x] Implement the GPU interface
  • [x] Implement requestAdapater and GPUAdapter interface
  • [x] Implement requestDevice and GPUDevice interface
  • [ ] Implement resource creation (textures, buffers, samplers)
  • [ ] Implement pipeline creation (bind groups, pipeline layouts, shader modules, computer/render pipelines)
  • [ ] Implement command interfaces (command encoders, queues)
  • [ ] implement fences
  • [ ] Implement compute functionalities
  • [ ] Implement graphic functionalities (draw calls, swap chain)
• [ ] Pass the WebGPU CTS.

Link to the modified compute example.

cc @jdm, @kvark

[zakorgy]

Some high-level design suggestions from @gterzian from https://github.com/servo/servo/pull/24708#discussion_r346662690:

• [x] I wonder if you really want to always start the webgpu component on start-up, rather you might want to start-it when it's first needed.
• [x] I wonder if you want any script to just be able to start using WebGPU whenever they want by calling methods on GPU inside script. Instead I think you want to mediate that capability via the constellation. So when script calls RequestAdapter, it should ideally go through the constellation via a ScriptMsg::RequestAdapter(sender, options), and be treated as an untrusted request for a specific capability. It ties into 1 in the sense that the constellation could then start a WebGPU instance, and respond to script with a dedicated means of communication with it(so subsequent messages don't all have to go through the constellation).
• [x] The spec doesn't mention it, however I wonder if you don't want some sort of map on the constellation like HashMap<EventLoop, WebGPU>, so that each event-loop(read: agent-cluster) has it's own instance. Perhaps its easier to initially only allow one, meaning reject any ScriptMsg::RequestAdapter if a script is already using WebGPU. While the spec says "Applications can hold onto multiple adapters at once"(https://gpuweb.github.io/gpuweb/#adapters), that doesn't necessarily means different origins or agent-cluster can use adapters at the same time. I assume one "application" is an agent-cluster, which would include the main page and any dedicated workers started by it. So one "application"could use multiple adapters, however I wonder if you want multiple applications running in parallel.
• [ ] I see a lot of those interfaces in the spec are Serializeable, and I wonder if that also shouldn't be restricted to agent-cluster. Shared array buffers are also restricted to agent-cluster, I believe. See https://html.spec.whatwg.org/#can-share-memory-with and https://tc39.es/ecmascript_sharedmem/shmem.html#WebBrowserEmbedding on the other hand I'm not sure if those objects actually "share memory" or not, or how they would be used concurrently by different agents. That's something that could be clarified in the spec, and to keep in mind as you implement the rest.
• [ ] It's going to be really interesting to make all these things Serializeable, and it will I think require the changes made in #24123.

[kvark]

fwiw, we do want multiple independent WebGPU applications to be able to run in parallel

[gterzian]

fwiw, we do want multiple independent WebGPU applications to be able to run in parallel

How would you define one "application"?

For example, I can imagine multiple agents, like a window and a dedicated worker, running in parallel but actually forming a single "application" and operating on the same (shared?) objects.

Then for example a cross-site iframe could be running another "application". It would be "in parallel", but also isolated in some way from the other application.

Then another example would be: can you share some webgpu obect with the cross-site iframe? Would the iframe and the main window then form a single application?

I think one way to express what an isolated application would be could be by limiting the serializablity of interfaces, using the upcoming infra from https://github.com/whatwg/html/pull/4940

You could also define some limitations in getting an adapter, so that for example multiple calls to get the same type of adapter within a given agent-cluster would return the same one(so that you can only have one application per agent-cluster, for example)

---

I have another question on something that is not clear to me from the spec: how would a webgpu application "run"? Will there be some equivalent of animation frame callbacks? I'm just wondering how the various interfaces will be used in practice, and how the JS would become aware of various stuff happening on the GPU.

For example the web audio spec defines a rendering-loop(which potentially calls into JS in a dedicated worklet).

I assume you'll define something similar? I assume there will be something requiring calling repeatedly into JS while the something like the GPU pipeline makes progress?

[zakorgy]

Some other things we should take care:

• [x] Replace wgpu-native with wgpu-core
• [x] Add identity management to the script side

[imiklos]

• [ ] We should also manage the resource id's release/destroy in servo side.