search

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
https://arxiv.org/abs/2210.17323
Apache License 2.0
1.81k stars 145 forks source link

Title: Feature Request: Add Saving Quantized Weights Functionality to bloom.py #12

Closed bestpredicts closed 1 year ago

bestpredicts commented 1 year ago

Description:

Hi there,

I noticed that the opt.py file in the repository provides a method for saving quantized weights, but this functionality is not available in the bloom.py file. I was wondering if it would be possible to add this feature to bloom.py as well.

Being able to save quantized weights is a really useful feature for optimizing the size of models, and it would be great to have this functionality available in all relevant files in the repository.

If this feature could be added to bloom.py, I think it would be a really helpful addition for anyone who is working with this file.

Thank you for your time and consideration.

Best regards,

efrantar commented 1 year ago

This was added with 64c61f6cae30cbc7d02dbe4e87b0cc32dd82b042.