Closed copybara-service[bot] closed 2 weeks ago
1.15x 7b sfp prefill speedup: Matmul in attention 2b bf16: prefill 114.456 -> 115.222 decode 16.8847 -> 16.9987
7b sfp: prefill 18.8575 -> 21.7325 decode 5.68428 -> 5.79791
1.15x 7b sfp prefill speedup: Matmul in attention 2b bf16: prefill 114.456 -> 115.222 decode 16.8847 -> 16.9987
7b sfp: prefill 18.8575 -> 21.7325 decode 5.68428 -> 5.79791