-
When I do QAT with my model, it's extremly slow when I want to trained with "per-channel" mode and modify the config.json like below. But It's fast with I set "per_channel_quantization"=False.
Can y…
-
# URL
- https://arxiv.org/abs/2210.17323
# Affiliations
- Elias Frantar, N/A
- Saleh Ashkboos, N/A
- Torsten Hoefler, N/A
- Dan Alistarh, N/A
# Abstract
- Generative Pre-trained Transformer …
-
Can you please explain what need to be changed for the following error ? Thank you.
python inference/inference_sim.py -a resnet50 -b 512
/home/user/anaconda3/lib/python3.7/site-packages/yaml/c…
-
I tried 50% log quantization on the pre-trained vgg16, however failed to re-gain the original accuracy.
Have anyone successful with the experiments?
Any suggestions on how to re-gain the accuracy …
-
Hi team,
Actually, I had trained yolov5 for custom object detection model and I had carried out compression techniques and it worked for me.
But quantization on yolov5 models is throwing error
-…
-
### Your current environment
```
(vllm-gptq) root@k8s-master01:/workspace/home/lich/QuIP-for-all# pip3 list | grep aphrodite
aphrodite-engine 0.5.3 /workspace/home/lich/aphrodite-eng…
-
### 💡 Your Question
Hi,
I am just checking, I see in the provided results that Yolo-NAS-L does not suffer much reduction in performance going to Yolo-NAS-INT8-L. Can I check what exactly is meant …
-
### Question
I'm trying to see what can run on an 8GB Raspberry Pi 5, and it occours to me that your approach might scale down really well. Any tips for replicating what you did with something like T…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussi…
-
So, i was trying to run this in google colab:
```
!python /content/multi_token/scripts/serve_model.py \
--model_name_or_path mistralai/Mistral-7B-Instruct-v0.1 \
--model_lora_path sshh12/M…