search

sign / translate

Effortless Real-Time Sign Language Translation
https://sign.mt
Other
489 stars 83 forks source link

Human Pose Viewer - Support WebGPU For Faster Inference #41

Closed AmitMY closed 2 years ago

AmitMY commented 2 years ago

Problem

Human pose viewer is quite slow, especially on lower end devices. This is because it is relying on the CPU, or high level API (WebGL).

Description

WebGPU is not currently supported by default in any major browser. It is a low level API, a few times faster, that should allow scaling the model from 256x256 to higher resolutions.

Alternatives

Human pose viewer inference on a server can be faster, with multiple high end GPUs, but I would like to have client-side support.

Additional context

https://caniuse.com/webgpu

AmitMY commented 2 years ago

Added webGPU and wasm support - https://github.com/sign/translate/commit/b72a778ef317a4b0a6f8a1c58b1894ba7a9b1b95

Alongside with other improvements, webgpu takes ~63ms compared to webgl which takes ~75ms on Macbook Pro 14 2021