issues
search
qwopqwop200
/
GPTQ-for-LLaMa
4 bits quantization of LLaMA using GPTQ
Apache License 2.0
2.97k
stars
455
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
fixes safe tensor compatibility and c4 dataset breaking
#290
a5hwinjs
opened
1 month ago
0
add support for minicpm
#289
LDLINGLINGLING
opened
1 month ago
0
GPTQ vs bitsandbytes
#288
iaoxuesheng
opened
4 months ago
0
Error when load GPTQ model
#287
KyrieCui
opened
6 months ago
0
datasets.utils.info_utils.ExpectedMoreSplits: {'validation'}
#286
SDcodehub
opened
7 months ago
1
Syntax changed in triton.testing.do_bench() causing error when running llama_inference.py
#285
prasanna
opened
8 months ago
0
Support Mistral.
#284
nbollman
opened
10 months ago
0
error: block with no terminator, has llvm.cond_br %5624, ^bb2, ^bb3
#283
Hukongtao
opened
11 months ago
0
neox.py needs to add "import math"
#282
StudyingShao
opened
1 year ago
0
LoRa and diff with bitsandbytes
#281
RonanKMcGovern
opened
1 year ago
0
Transformers broke again (AttributeError: 'GPTQ' object has no attribute 'inp1')
#280
EyeDeck
opened
1 year ago
1
Can i quantize HF version of llama model
#279
akanyaani
opened
1 year ago
0
Would GPTQ be able to support LLaMa2?
#278
moonlightian
opened
1 year ago
1
Why does the model quantization prompt KILLED at the end?
#277
g558800
opened
1 year ago
2
Help: Quantized llama-7b model with custom prompt format produces only gibberish
#276
Glavin001
opened
1 year ago
1
High PPL when groupsize != -1 for OPT model after replace linear layer with quantlinear.
#275
hyx1999
opened
1 year ago
1
Issue with GPTQ
#274
d0lphin
opened
1 year ago
1
can it support openllama model?
#273
Ted8000
closed
1 year ago
0
Could not obtain official perplexity using bloom_eval()
#272
xingyueye
opened
1 year ago
0
inference with the saved model error: AttributeError: module 'torch.backends.cuda' has no attribute 'sdp_kernel'
#271
LuciaIsFine
opened
1 year ago
2
llama_inference 4bits error
#270
gjm441
opened
1 year ago
0
Proposed changes to reduce VRAM usage. Potentially quantize larger models on consumer hardware.
#269
sigmareaver
opened
1 year ago
3
AttributeError: 'QuantLinear' object has no attribute 'weight' (t5 branch) (Google/flan-ul2)
#268
sigmareaver
closed
1 year ago
2
Update requirements.txt to prevent breaking
#267
nikshepsvn
closed
1 year ago
0
Update requirements.txt to prevent breaking
#266
nikshepsvn
closed
1 year ago
0
CUDA out of memory on flan-ul2
#265
sigmareaver
closed
1 year ago
1
SqueezeLLM support?
#264
nikshepsvn
opened
1 year ago
0
What is the right perplexity number?
#263
JianbangZ
opened
1 year ago
0
The detected CUDA version (12.1) mismatches the version that was used to compile PyTorch (11.7)
#262
siddhsql
opened
1 year ago
2
[Question] What is the expected discrepancy between simulated and actually computed values?
#261
set-soft
opened
1 year ago
4
Dependency conflicts for `safetensors`
#260
Yiximail
closed
6 months ago
1
Finetuning Quantized LLaMA
#259
Qifeng-Wu99
opened
1 year ago
0
fix typo in exception message
#258
gsaivinay
closed
1 year ago
0
compare with llama.cpp int4 quantize?
#257
luohao123
opened
1 year ago
0
changed reqs
#256
menandro
closed
1 year ago
0
How to quantize bloom after lora/ptuning?
#255
moonlightian
opened
1 year ago
0
I use python llama.py to generate a quantized model, but I can't find the .safetensors model
#254
jimi202008
closed
1 year ago
1
Wondering whether some of the triton or cuda kernel also speedup fp16 or not?
#253
drxmy
opened
1 year ago
0
the inference speed of GPTQ 4bit quantized model
#252
pineking
opened
1 year ago
2
Fix multigpu
#251
LQ1234
closed
1 year ago
0
Does this work for gptj specifically the cuda branch? Thanks!
#250
ArEnSc
opened
1 year ago
0
_pickle.UnpicklingError: invalid load key, 'v'.
#249
ahnHeejune
opened
1 year ago
1
Does not support 3bit quantization?
#248
foamliu
opened
1 year ago
0
An error is reported when running python setup_cuda.py install
#247
linuxdevopscn
opened
1 year ago
2
Errors encountered when running benchmark FP16 baseline on multiple GPUs
#246
foamliu
opened
1 year ago
2
No CUDA_ENV / conda-froce cudatoolkit-dev freezes
#245
nathanleclaire
closed
1 year ago
0
fused mlp is sometimes not working with safetensors, add an argument for it
#244
DalasNoin
closed
1 year ago
0
Sample code does not work
#243
foamliu
opened
1 year ago
2
Unable to run 'python setup_cuda.py install'
#242
ghost
opened
1 year ago
0
Build issue with newer torch pybind11 cast.h - workaround inside
#241
ilikenwf
opened
1 year ago
0
Next