search

cryscan / web-rwkv

Implementation of the RWKV language model in pure WebGPU/Rust.
Other
210 stars 15 forks source link

Multi-Node webGPU support #25

Open adminy opened 2 months ago

adminy commented 2 months ago

Cloudflare workers support webGPU, is there a way to take advantage of the fact that you get tons of worker nodes? I guess workerd also has to talk to the Vulkan / Metal API anyways but good thing that's still abstracted.

Btw Amazing work! It doesn't need benchmarks to tell its quite close to what the real hardware could give you.

cryscan commented 2 months ago

Thanks! I am currently working on a new version of the inference runtime which gets rid of waiting time in some CPU-bounded cases, and multi-device inferencing is on its roadmap!