search

HanGuo97 / flute

Fast Matrix Multiplications for Lookup Table-Quantized LLMs
https://arxiv.org/abs/2407.10960
Apache License 2.0
188 stars 6 forks source link

Add learnable scales functionality. #8

Closed radi-cho closed 2 months ago

radi-cho commented 2 months ago

Make possible to learn the scales from the FLUTE API.

Note: If we don't want flute to rely on datasets as well, we can remove the default corpus (wikitext-2) used now and document that providing a calibration corpus is mandatory.