Open vitonsky opened 3 years ago
CTranslate2 does not require a GPU. It can also run on CPU.
However, it uses external libraries for computation (such as oneDNN) that eventually rely on platform-specific instructions to run efficiently. As far as I know, WASM has incomplete support of these instructions, see e.g. https://github.com/WebAssembly/relaxed-simd/issues/9.
Currently we have no plans to explore this further, but any insight on this subject is welcome.
+1
I found an answer from Bing Chat which might be relevant here, but not sure whether it is valid as I hadn't verified it: TLDR: PyTorch -> ONNX -> WASM
To convert PyTorch models to ONNX format, you can use the torch.onnx.export()
function provided by PyTorch. Once you have an ONNX model, you can follow the steps outlined in the ONNX Runtime documentation to build ONNX Runtime for WebAssembly².
(2) Build for web | onnxruntime. https://onnxruntime.ai/docs/build/web.html.
Hi, in other issue to me say that probably i should will compile CTranslate2 to WASM
Source https://github.com/LibreTranslate/LibreTranslate/issues/120#issuecomment-888676870
I here because at this time i don't know how work CTranslate2 and see in README that it use GPU, then i have question.
Is possible use CTranslate2 for WASM or it will work not performant or even not work due to GPU bound?
What you think about it?