RuntimeError when quantize bloom using our code

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

MIT License

663 stars 50 forks source link

RuntimeError when quantize bloom using our code #29

Open Louym opened 10 months ago

Louym commented 10 months ago

We face a problem when we try to support Bloom models. The error occurs when we are trying to train the let. We write the int_bloom_layer.py according to the other three models but face this problem. The error lies in utils.py line 32