search

huggingface / ratchet

A cross-platform browser ML framework.
https://ratchet.sh
MIT License
542 stars 30 forks source link

WASM backend #146

Open FL33TW00D opened 4 months ago

FL33TW00D commented 4 months ago

Whilst WebGPU support is growing, perhaps it would be prudent to have a WASM backend to ensure 100% device coverage.

akashicMarga commented 3 months ago

@FL33TW00D have you checked this https://github.com/WasmEdge/WasmEdge ? they already have llaama.cpp, pytorch and tensorflow backend.

this project is for llm - https://github.com/LlamaEdge/LlamaEdge/tree/main

FL33TW00D commented 3 months ago

@FL33TW00D have you checked this https://github.com/WasmEdge/WasmEdge ? they already have llaama.cpp, pytorch and tensorflow backend.

this project is for llm - https://github.com/LlamaEdge/LlamaEdge/tree/main

Yeah I've seen this! I think we can't really use it, but good reference!

flatsiedatsie commented 3 months ago

Interesting project. How is it related to Transformers.js? Since both are a project of HuggingFace, correct?

On topic: I'm working on a project that uses multiple 'engines' to achieve this.

I try to choose models that run on both, and then invisibly switch engines for the user, depending on their browser.

Transformers.js runs all the smaller but vital other models, like voice recognition, TTS, OCR, embedding, etc.

FL33TW00D commented 3 months ago

Interesting project. How is it related to Transformers.js? Since both are a project of HuggingFace, correct?

On topic: I'm working on a project that uses multiple 'engines' to achieve this.

  • WebLLM when for WebGPU is available
  • Wllama when it's not.

I try to choose models that run on both, and then invisibly switch engines for the user, depending on their browser.

Transformers.js runs all the smaller but vital other models, like voice recognition, TTS, OCR, embedding, etc.

Ratchet is mostly experimental at the moment! Transformers.JS is production ready :)