search

PygmalionAI / aphrodite-engine

PygmalionAI's large-scale inference engine
https://pygmalion.chat
GNU Affero General Public License v3.0
606 stars 78 forks source link

feat: optimized layernorm kernels #398

Closed AlpinDale closed 1 month ago

AlpinDale commented 1 month ago

Should reduce latency in the layernorm by a fair bit.