vllm-project llm-compressor issues

vllm-project / llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Apache License 2.0

379 stars 28 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

fix build error for nightly

#145 dhuangnm opened 6 hours ago
2
fp8 config for original, default, default kv"

#144 horheynm opened 7 hours ago
1
update

#143 dsikka closed 1 day ago
1
Cholesky requires positive definite

#142 chintan-ushur opened 1 day ago
2
[KV Cache] kv-cache end to end tests

#141 horheynm opened 1 day ago
1
Fix recipe nested in configs

#140 kylesayrs opened 2 days ago
2
Fix README Issue

#139 robertgshaw2-neuralmagic closed 2 days ago
1
KeyError with torch.float8_e4m3fn

#138 Lue-C opened 3 days ago
1
[Usage] Can I do GPTQ with FP8 KV cache scheme?

#137 CharlesRiggins opened 4 days ago
2
GPTQ example README bug

#136 CharlesRiggins closed 2 days ago
1
Activation Ordering Tests

#135 kylesayrs closed 2 days ago
0
Using Quantized Models with vLLM CPU Backend

#134 miracatici opened 5 days ago
1
[KV-Cache] Make k_scale, v_scale as attributes of self_attn using HFCache

#133 horheynm opened 6 days ago
0
Run Compressed Tests

#132 Satrat closed 2 days ago
1
Fix transformer test conditions

#131 dsikka closed 6 days ago
1
AttributeError: 'QuantizationArgs' object has no attribute 'actorder' for in4 quantization example

#130 Justin-Uniphore closed 1 week ago
6
[Usage] Where to set the tokenizer in the sparse example?

#129 CharlesRiggins opened 1 week ago
0
Do we have a roadmap?

#128 CharlesRiggins closed 6 days ago
2
Fix markdown check test

#127 dsikka closed 1 week ago
0
Set `ready` label for transformer tests; add message reminder on PR opened

#126 dsikka closed 1 week ago
1
compressed-tensors main dependency for base-tests

#125 kylesayrs closed 1 week ago
0
Correct typo

#124 kylesayrs closed 2 days ago
0
Fix style, fix noqa

#123 kylesayrs closed 1 week ago
0
Fix style of examples

#122 kylesayrs closed 1 week ago
0
Activation Ordering Strategies

#121 kylesayrs closed 2 days ago
0
GPTQ Algorithm Cleanup

#120 kylesayrs closed 1 week ago
4
Auto-Infer `mappings` Argument for `SmoothQuantModifier` Based on Model Architecture

#119 rahul-tuli opened 1 week ago
0
add `ready` label to trigger transformers tests

#118 dsikka closed 1 week ago
1
Add vLLM e2e tests

#117 dsikka closed 1 week ago
0
demote recipe string initialization to debug and make more descriptive

#116 kylesayrs closed 1 week ago
0
SmoothQuant mappings tutorial

#115 rahul-tuli opened 1 week ago
0
Update finetune and oneshot tests

#114 dsikka closed 1 day ago
0
Add FP8 KV Cache quant example

#113 mgoin closed 1 week ago
0
[Usage] How to do KV cache quantization?

#111 CharlesRiggins closed 1 week ago
3
[Performance]: SmoothQuant quantization is too slow

#112 zxy1119 opened 1 week ago
1
convert model to FP8 error

#110 kuangdao opened 1 week ago
1
Struggling to quantize Llama-3.1-70b - OOM/linalg.Cholesky issue

#109 lulmer closed 1 week ago
3
add debug

#108 horheynm closed 1 week ago
0
SmoothQuant doesn't work with cpu offloading

#107 anmarques opened 1 week ago
1
[Bug]: Index Error tuple out of range

#106 SeanIsYoung opened 1 week ago
2
Yaml parsing fails with a custom mapping provided to SmoothQuantModifier recipe

#105 aatkinson opened 2 weeks ago
5
metrics

#104 horheynm closed 1 week ago
1
remove workflows + update runners

#103 dsikka closed 1 week ago
0
Pydantic Error

#102 CharlesRiggins closed 1 week ago
10
Enable obcq/finetune integration tests with `commit` cadence

#101 dsikka closed 2 weeks ago
0
Validate Recipe Parsing Output

#100 kylesayrs closed 3 hours ago
0
[ DOC ] Big Model Example

#99 robertgshaw2-neuralmagic closed 2 weeks ago
2
Update test

#98 dsikka closed 2 weeks ago
0
Update test config files

#97 dsikka closed 2 weeks ago
0
[ DOC ] Simple Example with Mistral

#96 robertgshaw2-neuralmagic opened 2 weeks ago
0