issues
search
turboderp
/
exllamav2
A fast inference library for running LLMs locally on modern consumer-class GPUs
MIT License
3.28k
stars
243
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
How to clear cache / reset the cache so that model doesnt remember the response earlier?
#212
Rajmehta123
closed
7 months ago
2
(Oobabooga) Can't load GPTQ models anymore with ExLlama-V2 0.0.10
#211
Daviljoe193
closed
7 months ago
2
Allow padding data instead of concatenating when generating calibration dataset
#209
ivsanro1
opened
7 months ago
1
Adding return_lowest_perplexity
#206
ziadloo
opened
7 months ago
0
Added draft model rope scale to chat example
#204
SinanAkkoyun
closed
7 months ago
0
Difference between gemm_half_q_half_gptq_kernel and gemm_half_q_half_kernel
#202
frankxyy
closed
7 months ago
0
Quantization error "Warning: Applied additional damping" and "Hessian"
#201
yamosin
closed
7 months ago
2
Generating a batch of different propmpt sizes, the shorter prompts tend to suffer
#200
ziadloo
closed
1 month ago
7
flash attention does nothing
#199
Tedy50
closed
2 months ago
5
Support for no_repeat_ngram_size
#198
anujnayyar1
opened
7 months ago
0
Support GPT2 tokenizer for CausalLM 72b
#196
CyberTimon
closed
7 months ago
6
Fix Unicode errors when loading files
#195
bdashore3
closed
7 months ago
0
Load models' tokenizer.json file with utf-8 encoding.
#194
VldmrB
closed
7 months ago
1
support for awq
#193
frankxyy
closed
1 month ago
0
Added DeepSeek Coder Instruct to chat example
#192
SinanAkkoyun
closed
7 months ago
0
Fixed code block syntax highlighting for long code
#191
SinanAkkoyun
closed
7 months ago
1
Chat code syntax highlighter has problems when out of view
#190
SinanAkkoyun
closed
7 months ago
4
Add Huggingface tokenzier support
#189
DOGEwbx
closed
7 months ago
13
Support for Huggingface Fast Tokenizers
#188
bibekyess
closed
7 months ago
13
Support LookaheadDecoding?
#187
yhyu13
closed
7 months ago
2
Batched generations are very similar
#186
anujnayyar1
closed
7 months ago
2
Any plans for Mac/Metal support?
#184
aikitoria
closed
6 months ago
3
How do I deal with custom EOS?
#183
teknium1
closed
7 months ago
1
fix infer websocket_actions.py
#182
Kerushii
closed
7 months ago
0
Different groupsizes for different bitwidths?
#181
dalistarh
closed
7 months ago
4
[Question] `h_gemm` Implementation
#180
jeromeku
closed
6 months ago
2
convert.py ending with "Killed" at lm_head layer when converting zephyr-7b
#179
Christopheraburns
closed
1 month ago
1
QuIP#, New SOTA(?) 2-bit Quantization Method
#176
brucethemoose
opened
8 months ago
6
Is there a colab or something which shows all the code necessary to setup the project?
#175
slooi
closed
7 months ago
1
EXL2 Proposal: Llamacpp Allocation Heuristics and Specialisation Degree Feature
#174
aljungberg
closed
1 month ago
7
Windows 10, Oobabooga, inability to load some models
#173
homeworkace
closed
7 months ago
3
Improve model load performance
#171
abstractdescutcheon
closed
6 months ago
4
Lora doesn't impact the model outputs
#169
matankley
closed
6 months ago
1
AssertionError: Insufficient space in device allocation
#168
Double-bear
closed
6 months ago
2
Update base.py to remove useless statement
#167
MilesQLi
closed
6 months ago
0
OOM with gpu_split_auto, must specify split manually
#166
cikkle
closed
1 month ago
8
Performance with speculative decoding is slightly worse than without at full context
#165
cikkle
closed
6 months ago
3
dynamic enable special tokens
#164
wangyu1997
closed
6 months ago
1
How can you clear the cache of the exllamav2?
#163
Rajmehta123
closed
2 months ago
2
CPU offloading?
#162
oobabooga
closed
7 months ago
1
self.rms_norm_eps = read_config["rms_norm_eps"] KeyError: 'rms_norm_eps' (Qwen model not supported)
#160
tutu329
closed
6 months ago
8
No module named 'exllamav2_ext' when loading a model
#159
ParisNeo
closed
2 months ago
11
exllamav2 Installation Error : [Errno 20] Not a directory: 'hipconfig'.
#158
watchstep
closed
7 months ago
6
Tweak to multiple cache example
#157
dvianisoho
closed
8 months ago
2
Got error when converting Llama2 70b
#156
GolemXlV
closed
8 months ago
2
Yi models generates some languages in an incorrect encoding
#155
samuelazran
closed
8 months ago
2
Simplify HIP compatibility
#154
ardfork
closed
8 months ago
8
Very poor performance when VRAM is nearly full (inconsistent)
#153
QM60
closed
3 months ago
13
Error When Converting Safetensor to exl2
#152
Noobville1345
closed
6 months ago
3
70B Quant Potential Issue
#151
azureblackprime
closed
8 months ago
2
Previous
Next