search

cloudflare / workerd

The JavaScript / Wasm runtime that powers Cloudflare Workers
https://blog.cloudflare.com/workerd-open-source-workers-runtime/
Apache License 2.0
6.27k stars 303 forks source link

WebGPU support for graphics rendering #1276

Open zegevlier opened 1 year ago

zegevlier commented 1 year ago

Hey :wave:,

I was very excited when I heard that Workers was getting WebGPU support, as I had wanted to use workers to render some 3d models. I was disappointed to hear that the APIs required for graphics rendering were not yet supported.

Are there plans to add support for these in the near future? If the issue with adding these APIs is time for the team members, would a PR that adds them be accepted? These missing APIs are part of the same standard as the rest of the WebGPU APIs, so adding support for them would round out the implementation of this standard in workerd.

Edit: I don't seem to be able to add labels to this issue, but I believe it should be labelled as a "feature proposal" as per the contributing guidelines.

jasnell commented 1 year ago

Our current focus is on enabling the WebGPU APIs that are needed primarily for AI applications, but will explore if graphics rendering would be possible. It's hard to say for sure.

NickAcPT commented 1 year ago

Just came across this and I wanted to say that I too was excited with the WebGPU support. That is, until I heard that it only supports the compute part of it. I too wanted to migrate to doing 3D rendering on the edge.

Until then, I have to resort to using software rendering (thanks to llvmpipe with the mesa-vulkan-drivers package).

I wonder if a good middle ground for when the Worker is ran somewhere without a GPU could be having a setting for "please use software-rendering if no gpu available, I understand it will be slow but I'm okay with the pain".

edevil commented 1 year ago

Hey @zegevlier. I was wondering what were you expecting to use as GPUCanvasContext in the context of a worker?

We are still coming up with use cases for the rendering part of WebGPU, so maybe if you can detail a bit what you wanted to do we can have a better idea. Also, can you list the missing APIs that you'd need?

Thanks.

zegevlier commented 1 year ago

Hey! Glad to hear this is being looked at.

I don't have a perfect understanding of WebGPU, but to my understanding, I would be able to render the result to an image without needing a GPUCanvasContext. This requires creating a texture that WebGPU will render to. I have a version of this running locally (on native rust, not workerd) using the wgpu rust crate. It is based on this tutorial, which is a bit outdated. By porting the shader code to wgsl I got it running in WASM on workerd up until it tried to use one of the missing WebGPU APIs.

I believe I would need all APIs listed in the not yet supported section of the documentation. My goal is to take .bbmodel files created in Blockbench (a low-poly 3D model editor) and render them using Workers. I already have the rust code working for this, and as far as I can tell it uses the wgpu equivalent of all those APIs.

Hope this helps. Please let me know if there is anything else I can do to help get this added!

edevil commented 1 year ago

Can you provide the converted shader code that you're using? I want to write a testcase that encompasses the missing functionality and I will base it on that "windowless" tutorial so that we can prioritize it.

NickAcPT commented 1 year ago

Can you provide the converted shader code that you're using? I want to write a testcase that encompasses the missing functionality and I will base it on that "windowless" tutorial so that we can prioritize it.

@edevil I'm not the OP, so apologies for hijacking the issue request. I am also interested in WebGPU 3d rendering with cf workers and I do have this shader that I use when rendering.

zegevlier commented 1 year ago

Sure! I just pushed a basic version to zegevlier/workerd-graphics-minimal. It's not very clean, and only uses some of the APIs, but this is the most minimal version. Please let me know if you want access to that .bbmodel renderer as well, but that is far less minimal.

edevil commented 11 months ago

I implemented the needed methods to be able to run this script: https://github.com/cloudflare/workerd/blob/main/src/workerd/api/gpu/webgpu-windowless-test.js

Is this enough for the use-case you mentioned? If there is still missing functionality that you think is interesting please provide a sample webgpu script, preferably in JS, and I'll take a look.

zegevlier commented 11 months ago

Awesome! I've been eagerly following the PRs, so I'm excited to hear that it's gotten so far.

I believe there are still some methods that I use that are not yet implemented. When I try to run my .bbmodel renderer, it complains that the .writeTexture function is not defined. As mentioned in the documentation, this is a convenience function, so it's probably possible to work around it. I haven't had the chance to prepare a minimal example or find a workaround yet.

Having a quick look at the methods I need vs the methods that are implemented, I see that .drawIndexed (https://developer.mozilla.org/en-US/docs/Web/API/GPURenderBundleEncoder/drawIndexed) and the associated setIndexBufferand setVertexBufferare also not implemented. I don't believe there are any reasonable workarounds to those, and these are quite useful for rendering more complex shapes.

I will spend some time this weekend preparing a JS example that uses all of these (and any others that I find that are needed).

Thank you for implementing everything you have! I can't wait to get this working 😄

chris-deep commented 8 months ago

Following this... I have a 3D app built on ThreeJS and it runs both as a web and desktop app. I have built POC to run the app on a GPU cloud and stream it to my desktop app (pixel / window streaming so the app can do 3D on the cloud GPU and not locally) which works fine. But, if CF builds an alternative I would use it... In fact, I'm probably going to build my own GPU colo servers across the US to offset/supplement what the GPU cloud provider offers me. I'll post here if / when I do bc others probably need it as well. But, hopefully CF one day offers this.

smallbraingames commented 4 months ago

Hey–I was wondering when WebGPU support will be enabled in prod?

alejg commented 3 months ago

Hey–I was wondering when WebGPU support will be enabled in prod?

I'm also interested. Tried but fails with the error: The compatibility flag webgpu is experimental and cannot yet be used in Workers deployed to Cloudflare.

gabe-burnette commented 2 months ago

Any update on WebGPU support in prod?