search

ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
23 stars 17 forks source link

Added Quantization Granularity of Matmul Inputs #229

Closed mmoffatt2 closed 2 months ago

mmoffatt2 commented 2 months ago

Added activation quantization options for qk matmul inputs and pv matmul inputs

Note: Should be merged in after or in replace of https://github.com/ReaLLMASIC/nanoGPT/pull/224, as it simplifies code from that PR