search

huggingface / ratchet

A cross-platform browser ML framework.
https://ratchet.sh
MIT License
627 stars 33 forks source link

Vectorize KV-Cache by using `Vec4` #222

Open officialcjunior opened 5 months ago

officialcjunior commented 5 months ago

Currently, the KV-Cache operation is scalar, this PR attempts to vectorize it.

Fixes #210

github-actions[bot] commented 5 months ago
Code Metrics Report 
  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 TOML                    1           75           63            2           10
-------------------------------------------------------------------------------
 Rust                   62        13276        11417          185         1674
 |- Markdown            34          311            0          244           67
 (Total)                          13587        11417          429         1741
===============================================================================
 Total                  63        13351        11480          187         1684
===============================================================================
FL33TW00D commented 4 months ago

@officialcjunior Haven't forgotten about this, just working on the refactor 👍🏻