search

xuyuzhuang11 / OneBit

The homepage of OneBit model quantization framework.
MIT License
157 stars 4 forks source link

could you provide opt1.3b.sh? #9

Closed zxbjushuai closed 3 months ago

zxbjushuai commented 3 months ago

I want to quantize llama2_7B but OOM.My cuda memory(48GB) is not enough for llama2_7B,so I want to try models that take up less memory

xuyuzhuang11 commented 3 months ago

Only a few parameters are changed here in LLAMA-7B.sh, you should change the model path with your OPT-1.3B path!

zxbjushuai commented 3 months ago

oh,thank you very much!🤗