search

NouamaneTazi / bloomz.cpp

C++ implementation for BLOOM
MIT License
812 stars 65 forks source link

Dramatically improve performance on iOS #23

Closed pcuenca closed 1 year ago

pcuenca commented 1 year ago

By simply using the appropriate compiler settings and enabling optimization.

Goes from ~960ms to ~30ms per token on Simulator. Runs at ~37ms per token on my iPhone 13 Pro.