search

huggingface / ratchet

A cross-platform browser ML framework.
https://ratchet.sh
MIT License
627 stars 33 forks source link

CPU Backend #258

Open FL33TW00D opened 1 month ago

FL33TW00D commented 1 month ago

The Ratchet CPU backend is nearly here!

Some remaining work to be done:

  1. We have a "conflicting" execution model now between CPU and GPU. GPU is applied lazily (i.e we construct a compute graph of all operations and then dispatch to the GPU, for CPU we apply as we build the graph).
  2. Many operations not yet implemented.
  3. Matmul optimizations required for performance to be acceptable. Candle did well here with some wasm128 magic.