The ckpt of Quantized OPT model is not be found

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

MIT License

626 stars 49 forks source link

The ckpt of Quantized OPT model is not be found #53

Open liuxy1103 opened 6 months ago

liuxy1103 commented 6 months ago

We have tried to reproduce the quantization of OPT-30B， but it is still difficult for us. Would you mind releasing the details of the procedure?

ChenMnZ commented 6 months ago

We have released the ckpt of OPT. You can find them at https://huggingface.co/ChenMnZ/OmniQuant/tree/main.

liuxy1103 commented 6 months ago

The results by loading the ckpt is not consistent with the reported result in the paper.

ChenMnZ commented 6 months ago

Can you reproduce the results of other models except OPT-30B？

linloong commented 2 weeks ago

When reproducing evaluation results for OPT-13b-w4a4, I got nan for wikitext2 dataset.

It seems that the checkpoint has also broken now.

And when I try to train by myself, the ppl is very high.

ChenMnZ commented 2 weeks ago

@linloong Can you provide the training script?

linloong commented 2 weeks ago

Sure, Actually, I just use the script you provided in the scripts/opt/opt-13b/w4a4.sh.

CUDA_VISIBLE_DEVICES=0 python main.py \
--model facebook/opt-13b  \
--epochs 20 --output_dir ./log/opt-13b-w4a4 \
--wbits 4 --abits 4 --lwc --let --alpha 0.75

@ChenMnZ