-
Hi,
I’m using YOLOv9 for segmentation tasks and noticed that quantization is currently supported for object detection models. Since the backbone is the same across all YOLOv9 variants, I wanted to …
-
https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
https://github.com/vllm-project/llm-compressor/tree/main/examples/quantization_w8a8_fp8
https://github.com/vllm-project/llm-compressor/tre…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and found no similar bug report.
### Ultralytics YOLO Component
_No …
-
Hi,
I would like to inquire about the support for HQQuant (HQQ) quantization with specific models. I am particularly interested in knowing if the following models are supported for HQQ quantization…
-
# Quantization Impact on Model Accuracy | Slightwind
Mistral-7B’s performance on 5-shot MMLU 如果对测试细节不感兴趣,只需要看下面给出的汇总表格即可。
Overview 量化/非量化版本的 Mistral-7B-v0.1 模型在 5-shot MMLU 上的表现:
Quant Type Compute D…
-
I really do not know much about the AI world and the limitations, but if this model can convert to 1.58, maybe it will make this model more accessible?
-
### 🚀 The feature, motivation and pitch
I'm working on applications that must run locally in resource-limited HW. Threrefore, quantization becomes essential. Such applications need from multimodal vi…
-
### System Info
a100
### Who can help?
@Tracin
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [x] An officially supported task in the `examples` …
-
**Describe the bug**
When I use llm-compressor to quantize llava model, but at the begining, it failed. (Unrecognized configuration class: 'transformers.models.llava.configuration_llava.LlavaConfig'…
-
i have completed stable diffusion quantization in txt2img as demo shows.
the result is very good.
when i want to transfer sd quantization in inpainting task, i meet the problem that the quantization r…