-
Hi,
I was trying out the compression library for ZeroQuant quantization (for GPT-J model). While I was able to compress the model, I didn't see any throughput/latency gain from the quantization dur…
-
I have used PTQ for int8 export from pytorch model and despite attempts at calibration, there is a significant drop in detection accuracy.
I am moving to quantization aware training to improve the…
-
Was chatting with @Chillee about our plans in AO today and he mentioned we should be focusing on a few concrete problems like
1. Demonstrate compelling perf for fp8 gemm at a variety of batch sizes.
…
-
try to run models/quickstart/image_recognition/tensorflow/resnet50/inference; both fp32 and int8.
Using wget command to download the .gz file to Documents directory and using tar to unzip the file.
…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
**For int8 inference case we are facing some issues and we tried to confirm with the reproducer.**
And we checked with different combinations, all are passing in the float32 cases and for int8 some a…
-
### Search before asking
- [X] I have searched the YOLOv5 [issues](https://github.com/ultralytics/yolov5/issues) and found no similar feature requests.
### Description
Exporting my model in…
-
### OpenVINO Version
2024.0.0
### Operating System
Ubuntu 22.04 (LTS)
### Device used for inference
CPU
### OpenVINO installation
PyPi
### Programming Language
Python
### Hardware Architectu…
-
**Describe the bug**
When inference bloom model with `replace_with_kernel_inject = True` , and `dtype = torch.int8`
For the reason that this model is trained by torch , I load the weight with tor…
-
```
python run_experiments.py 8 vit_b ../ ../../segment-anything experiments_data --run-experiments --num-workers 8 --capture_output True
```
there wasn't an error traceback, just the results for…