-
I've begun work on the ability to edit models and import new models. Here are the remaining features I'd like to complete:
- [x] Write out model files - necessary for everything else
- [ ] Model s…
-
### 🚀 The feature, motivation and pitch
I'm working on applications that must run locally in resource-limited HW. Threrefore, quantization becomes essential. Such applications need from multimodal vi…
-
### 🚀 Feature request
Quantization is a widely used technique to accelerate models, particularly when using the [torch.compile](https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.htm…
-
i have completed stable diffusion quantization in txt2img as demo shows.
the result is very good.
when i want to transfer sd quantization in inpainting task, i meet the problem that the quantization r…
-
[Block-wise quantization](https://arxiv.org/abs/2110.02861) divides input tensors into smaller blocks that are independently quantized, resulting in faster optimization and high precision quantization…
-
### 1. System information
- Windows 11
- TensorFlow installation (pip package or built from source): pip
- TensorFlow library : 2.13
I am attempting to convert a QAT model trained with int8 we…
-
what was the quantisation algorithm used in unsloth/Llama-3.2-1B-bnb-4bit model: https://huggingface.co/docs/transformers/main/en/quantization/overview. Is it int4_awq or int4_weightonly ?
-
### System Info
GPU: 4090
Tensorrt: 10.3
tensorrt-llm: 0.13.0.dev2024081300
### Who can help?
@Tracin May you please have a look, thank you very much
### Information
- [ ] The official example sc…
-
### 🚀 The feature, motivation and pitch
With a single command, quantize the same model across every available quant scheme and configuration and output a table that compares the results. This will …
byjlw updated
4 weeks ago
-
### System Info / 系統信息
torch 2.5.1+cu121
diffusers 0.31.0
torchao 0.7.0+cpu
Python 3.11.10
Windows 11
### Information / 问题信息
- [X] The official example scr…