-
### bug描述 Describe the Bug
在 PaddleSlim PTQ量化后导出的模型在进行 Paddle Inference 的 int8 推理的时候会报如下所示的错误:
![image](https://github.com/PaddlePaddle/Paddle/assets/69797242/80b898ae-ef6e-4226-8412-8cc1dfff8e37)
…
-
what does int8 calibration really do?
is it PTQ? or something else?
-
Thanks for the contribution of the author.
I follow the inference script to run the txt2img demo:
```
python scripts/txt2img.py --prompt "a puppy wearing a hat" --plms --cond --ptq --weight_bit…
-
## 📝 Description
While going through your YouTube video explanation on Quantisation. I came across this doubt when I was validating the formulas of `scales` and `zero_point` for Asymmetric and Symmet…
-
您好,我看教程里面只有使用rknn自带的工具做ptq量化,如果使用外部工具做好qat或者ptq量化,然后得到量化参数,请问rknn怎么加载这个onnx网络和对应的量化参数?
-
### 💡 Your Question
I used the [Roboflow notebook](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/train-yolo-nas-on-custom-dataset.ipynb?ref=blog.roboflow.com) to …
-
Hello,
It seems that currently int8 weight only and SmoothQuant quantizations are supported for GPT models, but no kind of quantization is supported for other autoregressive transformer models, suc…
-
Hi neuralmagic team !
Very nice work with AutoFP8 ! We were thinking of integrating AutoFP8 in transformers, so that users can run your checkpoints directly with transformers ! We would simply rep…
-
I use the AIMET PTQ to quantize the CLIP text model.
But I encounter this error [KeyError: 'Graph has no buffer /text_model/encoder/layers.0/layer_norm1/Constant_output_0, referred to as input for …
-
Hello,
I am currently experimenting with the sample codes provided in the documentation/examples. I'm utilizing the default ResNet50 model, along with the dataset loader and evaluator functions. Cu…