Closed austinvhuang closed 1 month ago
Thanks! @pculliton @samkaufman FYI. We'll soon have a basic MatMul to test with.
Related reading: https://siboehm.com/articles/22/Fast-MMM-on-CPU Which links to https://marek.ai/matrix-multiplication-on-cpu.html and https://github.com/flame/how-to-optimize-gemm/ (from the BLIS group).
This is now done :D
Call for contributions for anyone interested in taking this on (@jan-wassenberg feel free to tag anyone who might be interested). The Prefill() computation is setup to allow batched computation (currently statically sized as
kPrefillBatchSize
).Some pointers:
kPrefillBatchSize > 1
kBatchSize
is known at comptime, this could probably even be done withif constexpr
FFW()
and assess performance differences since there's less implementation complexity to deal with.