search

finalfusion / finalfrontier

Context-sensitive word embeddings with subwords. In Rust.
https://finalfusion.github.io/finalfrontier
Other
87 stars 4 forks source link

Add ARM Neon implementations of dot/scale/scaled_add #180

Closed danieldk closed 2 years ago

danieldk commented 2 years ago

Since Neon intrinstics require nightly features, the vectorized functions are only enabled on Rust-nightly.

The speedup depends a bit on the model. The scaled_add function did already get vectorized with recent Rust compilers. However, the same is not true for dot, probably because it is unsafe math (floating point summation order).

Dot product before this change:

dot-without-neon

After this change:

dot-with-neon