-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch…
-
A new, interesting quantization scheme was published, which not only reduces memory consumption (like current quantization schemes), but als reduces computations.
> **[QuaRot: Outlier-Free 4-Bit In…
-
### System Info
CPU x86_64
GPU NVIDIA L20
TensorRT branch: v0.13.0
CUDA: NVIDIA-SMI 535.161.07 Driver Version: 535.161.07 CUDA Version: 12.5
### Who can help?
@kaiyux @byshiue
### Information…
-
### 🐛 Describe the bug
JIT tracing a quantized model that has forward_pre_hooks throws the following error:
`RuntimeError: Couldn't find method: 'forward' on class: '__torch__.torch.ao.nn.intri…
-
I encountered an issue while trying to quantize the YOLOv8s model using the Ryzen AI quantizer. Below are the details of the error:
### Error Message:
```
No CUDA runtime is found, using CUDA_HOM…
-
原模型输出结构:
![图片](https://github.com/user-attachments/assets/1572728b-d965-4bf3-8c19-aed8266f35c3)
onnx_edit后结构:
![图片](https://github.com/user-attachments/assets/b5a1d2f6-b71c-4fbe-b71b-9408291a0e49…
-
Trying to quantise some flux models to lower the vram needs and I get that error.
```
(venv) C:\AI\llama.cpp\build>bin\Debug\llama-quantize.exe "C:\AI\ComfyUI_windows_portable\ComfyUI\models\chec…
-
I have the same problem like this [cannot use unsloth](https://github.com/unslothai/unsloth/issues/820), but when I run the code below it is still got the same error :
`os.environ['CUDA_VISIBLE_DEVIC…
-
### Describe the issue
When trying to quantize a Yolov8 model (exported with `yolo export model=yolov8x.pt format=onnx`) with `onnxruntime`, I get the following error:
```
$ python quantize.py yo…
Jamil updated
3 months ago
-
### System Info
A100-80G
cuda12.1
bitsandbytes 0.43.2.dev0
diffusers 0.29.1
lion-pytorch 0.2.2
torch 2.0.1
torch-tb-profiler 0…