-
E:\ai-toolkit>python run.py first2
Running 1 job
Error running job: Could not find config file first2
========================================
Result:
- 0 completed jobs
- 1 failure
=======…
-
After quantizing an AutoModelForSequenceClassification model using autofp8, I observed a slight drop in performance. The left chart shows the inference time for bf16 linear layers, while the right cha…
-
Hi, here is the INC team from Intel. Thank you for developing this amazing project.
### Motivation
Our team has developed Auto-Round, a new weight-only quantization algorithm. It has achieved …
-
It would be great to get the instructions to run the 3B model locally on a gaming GPU (e.g. 3090/4090 with 24GB VRAM).
### Confirmed GPUs
From this thread
| GPU Model | VRAM (GB) | Tuned-3b | T…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
Wasn't sure if this was technically a new issue but just in case I'm reposting here:
I figured out how to dynamically quantize the instructor-xl model, but at the point that it's supposed to create…
-
Executorch issue:
https://github.com/pytorch/executorch/issues/3588
https://github.com/pytorch/torchchat/actions/runs/9047866134/job/24860312456?pr=751
```
+ python3 torchchat.py export storie…
-
**Describe the bug**
Quantizing mlp.down_proj in layer 0 of 125: 0%| | 0/126 [00:44
-
I was going through the **[Clip Zero-Shot Image Classification](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/228-clip-zero-shot-image-classification)** section and I repli…
-
Hello. I have a question about `oc_batch_size`.
https://github.com/casper-hansen/AutoAWQ/blob/63d2aaec7b3849eadd6fee8df767cf92c30ee65c/awq/quantize/quantizer.py#L262
As you can see above, `oc_batch_…