OpenGVLab OmniQuant issues

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

MIT License

663 stars 50 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

The llama-2-7b model can't quant in this code

#93 Hzqskywkr opened 8 hours ago
0
How to generalize LET to llama3?

#92 zjq0455 closed 18 hours ago
0
Error when evaluating MMLU

#91 zjq0455 opened 2 days ago
0
how to enable llama3-8b int4 awq models

#90 FlexLaughing opened 1 week ago
0
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat2 in method wrapper_CUDA_bmm)

#89 mcpaulgeorge opened 1 month ago
3
The version of transformers, auto_gptq, autoawq

#88 zhangfzR opened 1 month ago
0
Quantify tinyllama-1.1B-Chat-v1.0, a CUDA assertion error occurred

#87 causeof-you opened 1 month ago
0
[New Feature] Seek MLA Supported by Smooth

#86 RanchiZhao opened 1 month ago
0
question about let

#85 mxjmtxrm opened 2 months ago
0
[Model Request] MiniCPM

#84 RanchiZhao opened 2 months ago
0
The llama-1-65b model seems unstable in this code

#83 Xingrun-Xing opened 2 months ago
2
Questions about quantization

#82 mxjmtxrm closed 2 months ago
0
Questions about quantization

#81 mxjmtxrm opened 2 months ago
0
How to accelerate the inference speed with real_quant

#80 j2kim99 opened 3 months ago
3
Which bug do you fix for auto_gptq

#79 BaohaoLiao opened 3 months ago
1
Some questions about the results of weight only quantification in the paper

#78 everloom closed 4 months ago
0
Questions regarding Infusing Omniquant into MLC

#77 BuildBackBuehler opened 4 months ago
3
OPT-30B

#76 Arthur-Ling opened 4 months ago
0
Llama-3-8B

#75 hsb1995 opened 4 months ago
5
Is activation get quantized on-the-fly?

#74 XA23i closed 4 months ago
5
Why is the compressed file one file instead of the pre trained weights, where there are many files for training the mode

#73 hsb1995 opened 4 months ago
1
TypeError: FalconRotaryEmbedding.forward() missing 1 required positional argument: position_ids

#72 luchangli03 opened 4 months ago
0
AttributeError: 'FalconAttention' object has no attribute 'maybe_rotary'

#71 luchangli03 opened 4 months ago
1
W4A4 in llama2-7b

#70 chenzx921020 closed 4 months ago
4
When reproducing evaluation results for Llama-2-13b w4a4, I got nan

#69 NewDriverLee closed 2 months ago
4
KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'

#68 zfstr closed 4 months ago
2
Other Task

#67 hsb1995 opened 5 months ago
1
seq_len is deprecated and unused in transformers>=4.38.0

#66 Lokshaw-Chau closed 4 months ago
1
Checksums didn't match for dataset source files

#65 hsb1995 closed 4 months ago
7
RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed).

#64 zkf331 closed 6 months ago
3
OPT Model Reproduction Discrepancies

#63 fantasysee closed 6 months ago
2
CUDA extension not installed

#62 Arthur-Ling closed 4 months ago
2
Difference between fake quant and real quant

#61 YihengBrianWu closed 4 months ago
1
reproduce evaluation results

#60 oujieww opened 7 months ago
9
How to properly evaluate W6A6 models using checkpoint from the mode zoo

#59 ChengZhang-98 opened 7 months ago
2
[WIP][quantize] add gptq post-quantization

#58 xingchensong opened 7 months ago
0
AutoGPTQ or AutoGPTQ-bugfix?

#57 Alvant opened 7 months ago
7
[quantizer] add Odyssey-style symmetric quantization

#56 xingchensong closed 7 months ago
2
License

#55 fakerybakery closed 7 months ago
2
[datautils] fix c4 dataset

#54 xingchensong closed 7 months ago
2
The ckpt of Quantized OPT model is not be found

#53 liuxy1103 opened 7 months ago
6
Quantize Llama-2-Chat Models with Weights and Activation-Quantization

#52 DRXD1000 closed 7 months ago
2
[Llama-2-7B-chat] ppl of w4a8 is nan

#51 xingchensong closed 8 months ago
4
How to use AutoGPTQ to achieve real quantization?

#50 AboveParadise closed 6 months ago
3
Bugfix/attention mask and implementation

#49 Alvant closed 8 months ago
1
[fix] 'QuantLlamaDecoderLayer' object has no attribute 'model_attn'

#48 xingchensong closed 8 months ago
1
[fix] attention_mask may appear None for newer versions of LLaMA

#47 xingchensong closed 8 months ago
1
attention_mask may appear None for newer versions of LLaMA?

#46 Alvant closed 8 months ago
3
[Model Request] upstage/SOLAR-10.7B-v1.0

#45 joseph777111 closed 5 months ago
1
TypeError: QuantLlamaDecoderLayer.forward() got an unexpected keyword argument 'padding_mask'

#44 xianwujie closed 8 months ago
1