-
### Description
Propose to integrate quantization methods into nano to reduce the model size and accelerate inference. [Neural Compressor](https://github.com/intel/neural-compressor) provides a set o…
-
## ❓ Question
I did quantize to FP16 by using python. but i didn't find way to do that with int8
let me know if there is support
-
**Describe the bug**
When converting a quantised model with input names to TFLite, setting `converter.inference_input_type == tf.int8` leads to different names for the inputs.
**System information…
-
为使您的问题得到快速解决,在建立Issues前,请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
如果您没有查询到相似问题,为快速解决您的提问,建立issue时请提供如下细节信息:
- 标题:简洁、精准概括您的问题,例如“Insufficient Memory xxx" ”
- 版本、环境信息:
1)PaddlePaddle版本…
-
I recently run post training quantization (PTQ) on BERT Base model for SQuADv1.1 task and fail to achieve the f1-score released at [this page](https://github.com/NVIDIA/FasterTransformer/tree/main/ber…
-
Hi! I recently tried to deploy [yolov5](https://github.com/zye1996/Yolov5_EdgeTPU) onto EdgeTPU with C++. The application runs fine now although the inference speed is not even comparable to the Pytho…
-
Run all and running inference for the output of the TF2 model, on a Coral TPU board always gives 0.00 accuracy, even for images from the training dataset.
Using the same input images on the TF1 c…
-
I recently run PTQ with bert-tf-quantization tool. Following FasterTransformer [README](https://github.com/NVIDIA/FasterTransformer), I have tensorflow==1.14.0 in my environment, which should be soppo…
-
Describe the bug
當我使用int 8的tflite進行kmodel轉換時,可以成功轉換,但無法在k210上運行
但當我使用float32的tflite進行kmodel轉換時,卻產生了error message: Fatal: inputs are not compatible to concat,無法進行轉換
這令我很不解
Environment (please…
-
**Describe the issue**
**In TF2, the full-integer quantized models produced by the TFLite Converter can only have *float* input and output type. This is a blocker for users who require *int8* or *uin…