qwopqwop200 GPTQ-for-LLaMa issues

qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ

Apache License 2.0

2.97k stars 455 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

fixes safe tensor compatibility and c4 dataset breaking

#290 a5hwinjs opened 1 month ago
0
add support for minicpm

#289 LDLINGLINGLING opened 1 month ago
0
GPTQ vs bitsandbytes

#288 iaoxuesheng opened 4 months ago
0
Error when load GPTQ model

#287 KyrieCui opened 6 months ago
0
datasets.utils.info_utils.ExpectedMoreSplits: {'validation'}

#286 SDcodehub opened 7 months ago
1
Syntax changed in triton.testing.do_bench() causing error when running llama_inference.py

#285 prasanna opened 8 months ago
0
Support Mistral.

#284 nbollman opened 10 months ago
0
error: block with no terminator, has llvm.cond_br %5624, ^bb2, ^bb3

#283 Hukongtao opened 11 months ago
0
neox.py needs to add "import math"

#282 StudyingShao opened 1 year ago
0
LoRa and diff with bitsandbytes

#281 RonanKMcGovern opened 1 year ago
0
Transformers broke again (AttributeError: 'GPTQ' object has no attribute 'inp1')

#280 EyeDeck opened 1 year ago
1
Can i quantize HF version of llama model

#279 akanyaani opened 1 year ago
0
Would GPTQ be able to support LLaMa2?

#278 moonlightian opened 1 year ago
1
Why does the model quantization prompt KILLED at the end?

#277 g558800 opened 1 year ago
2
Help: Quantized llama-7b model with custom prompt format produces only gibberish

#276 Glavin001 opened 1 year ago
1
High PPL when groupsize != -1 for OPT model after replace linear layer with quantlinear.

#275 hyx1999 opened 1 year ago
1
Issue with GPTQ

#274 d0lphin opened 1 year ago
1
can it support openllama model?

#273 Ted8000 closed 1 year ago
0
Could not obtain official perplexity using bloom_eval()

#272 xingyueye opened 1 year ago
0
inference with the saved model error: AttributeError: module 'torch.backends.cuda' has no attribute 'sdp_kernel'

#271 LuciaIsFine opened 1 year ago
2
llama_inference 4bits error

#270 gjm441 opened 1 year ago
0
Proposed changes to reduce VRAM usage. Potentially quantize larger models on consumer hardware.

#269 sigmareaver opened 1 year ago
3
AttributeError: 'QuantLinear' object has no attribute 'weight' (t5 branch) (Google/flan-ul2)

#268 sigmareaver closed 1 year ago
2
Update requirements.txt to prevent breaking

#267 nikshepsvn closed 1 year ago
0
Update requirements.txt to prevent breaking

#266 nikshepsvn closed 1 year ago
0
CUDA out of memory on flan-ul2

#265 sigmareaver closed 1 year ago
1
SqueezeLLM support?

#264 nikshepsvn opened 1 year ago
0
What is the right perplexity number?

#263 JianbangZ opened 1 year ago
0
The detected CUDA version (12.1) mismatches the version that was used to compile PyTorch (11.7)

#262 siddhsql opened 1 year ago
2
[Question] What is the expected discrepancy between simulated and actually computed values?

#261 set-soft opened 1 year ago
4
Dependency conflicts for `safetensors`

#260 Yiximail closed 6 months ago
1
Finetuning Quantized LLaMA

#259 Qifeng-Wu99 opened 1 year ago
0
fix typo in exception message

#258 gsaivinay closed 1 year ago
0
compare with llama.cpp int4 quantize?

#257 luohao123 opened 1 year ago
0
changed reqs

#256 menandro closed 1 year ago
0
How to quantize bloom after lora/ptuning?

#255 moonlightian opened 1 year ago
0
I use python llama.py to generate a quantized model, but I can't find the .safetensors model

#254 jimi202008 closed 1 year ago
1
Wondering whether some of the triton or cuda kernel also speedup fp16 or not?

#253 drxmy opened 1 year ago
0
the inference speed of GPTQ 4bit quantized model

#252 pineking opened 1 year ago
2
Fix multigpu

#251 LQ1234 closed 1 year ago
0
Does this work for gptj specifically the cuda branch? Thanks!

#250 ArEnSc opened 1 year ago
0
_pickle.UnpicklingError: invalid load key, 'v'.

#249 ahnHeejune opened 1 year ago
1
Does not support 3bit quantization?

#248 foamliu opened 1 year ago
0
An error is reported when running python setup_cuda.py install

#247 linuxdevopscn opened 1 year ago
2
Errors encountered when running benchmark FP16 baseline on multiple GPUs

#246 foamliu opened 1 year ago
2
No CUDA_ENV / conda-froce cudatoolkit-dev freezes

#245 nathanleclaire closed 1 year ago
0
fused mlp is sometimes not working with safetensors, add an argument for it

#244 DalasNoin closed 1 year ago
0
Sample code does not work

#243 foamliu opened 1 year ago
2
Unable to run 'python setup_cuda.py install'

#242 ghost opened 1 year ago
0
Build issue with newer torch pybind11 cast.h - workaround inside

#241 ilikenwf opened 1 year ago
0