search

laugh12321 / TensorRT-YOLO

🚀 TensorRT-YOLO: Supports YOLOv3, YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv9, YOLOv10, and PP-YOLOE using TensorRT acceleration with EfficientNMS, CUDA Kernels and CUDA Graphs!
https://github.com/laugh12321/TensorRT-YOLO
GNU General Public License v3.0
383 stars 50 forks source link

[Question]: Export .pt to .onnx with fp16 was removed #7

Closed deadmerc closed 4 months ago

deadmerc commented 4 months ago

Hello

I am only starting learning technology, trying to understand why half precision was removed from export and replaced with dynamic shape? They can work together or not?

laugh12321 commented 4 months ago

After practical testing, currently only YOLOv5 does not support exporting both half precision and dynamic shape simultaneously. Refer to https://github.com/ultralytics/yolov5/issues/11683 for more details.

The decision to deprecate the export of half precision in ONNX is because ONNX is commonly used as an intermediate representation for model migration. By providing more options when converting to other formats such as TensorRT, it allows for greater flexibility.

deadmerc commented 4 months ago

Thank you for quick answer, but how I understood if onnx not in fp16 then converting to tensorrt model with /usr/src/tensorrt/bin/trtexec --onnx=... --saveEngine=... --fp16 will not work ( output will be as fp32 ) right?

laugh12321 commented 4 months ago

When exporting an ONNX model in FP32, you can use the --fp16 option with trtexec to convert the precision to FP16 during the export of the TensorRT engine (default is FP32). However, when exporting an ONNX model in FP16, it is necessary to include the --fp16 option. For more guidance on using trtexec, please refer to https://github.com/NVIDIA/TensorRT/tree/main/samples/trtexec.

deadmerc commented 4 months ago

Understand, thank you