search

pytorch-labs / gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
BSD 3-Clause "New" or "Revised" License
5.57k stars 507 forks source link

improving GPTQ defauls #104

Closed HDCharles closed 7 months ago

HDCharles commented 7 months ago

Stack from ghstack (oldest at bottom):

Summary: previously wikitext was default task for GPTQ which with other defaults, wouldn't collect any examples. also improved error message.

Test Plan:

python quantize.py --checkpoint_path checkpoints/$MODEL_REPO/model.pth --mode 8da4w-gptq --calibration_limit 5 python quantize.py --checkpoint_path checkpoints/$MODEL_REPO/model.pth --mode 8da4w-gptq --calibration_limit 5 --calibration_tasks hellaswag

Reviewers:

Subscribers:

Tasks:

Tags: