ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
24 stars 18 forks source link

Added Lambda "Quantization Level" Parameter #287

Closed mmoffatt2 closed 3 weeks ago

mmoffatt2 commented 1 month ago

Adds a quantization level parameter to allow for more gradual introduction of quantization rather than applying it abruptly to the weights and activations for each tensor. Quantization Level = 0 means that there is no quantization happening. Quantization Level = 1 means full quantization. This quantization level is only applied during training to the activations and linears.

gkielian commented 1 month ago

From discussion change kwargs to iter_num directly.

And also provide a demo script that shows difference with and without gradual quantization with tinystories (verifying inference too)