search

WongKinYiu / YOLO

An MIT rewrite of YOLOv9
MIT License
477 stars 50 forks source link

Would you support conversion from torch to onnx and ncnn? #46

Open stereomatchingkiss opened 1 month ago

stereomatchingkiss commented 1 month ago

As the title mentioned, one of the strength of yolov9 is the relatively high accuracy with smaller size and faster speed, a great tool for embedded devices. I think it would be a nice feature for this project

ramonhollands commented 1 month ago

@henrytsui000 I would like to contribute in this one. Is there any work done on exporting to different formats like onnx, coreml and tflite?

henrytsui000 commented 4 weeks ago

@henrytsui000 I would like to contribute in this one. Is there any work done on exporting to different formats like onnx, coreml and tflite?

Thanks a lot!

You may find some existing code here: https://github.com/WongKinYiu/YOLO/blob/dc88787a7f6fff89d6e60fb63ee20a4be1721b34/yolo/utils/deploy_utils.py#L11

To be honest, I'm not sure if the code is robust enough, but you can activate it using the following command:

python yolo/lazy.py task=inference task.fast_inference=onnx

Currently, it only supports ONNX and TensorRT.

If you're willing, you can add support for CoreML and TFLite, and help make the code more robust.

Best regards,
Henry Tsui

ramonhollands commented 4 weeks ago

Thanks for your reply. Gonna check and try to contribute on this next weeks!

ramonhollands commented 4 weeks ago

@henrytsui000 I think we want to remove the auxiliary branch on all export formats, right?

'''

if self.compiler == "onnx": return self._load_onnx_model(device) elif self.compiler == "trt": return self._load_trt_model().to(device) elif self.compiler == "deploy": self.cfg.model.model.auxiliary = {}

'''

henrytsui000 commented 3 weeks ago

Yes, the auxiliary header is only used to train the model

ramonhollands commented 1 day ago

@henrytsui000

Thanks for your reply. I managed to export tflite and now in the progress in getting all things on GPU.

For that, can we get rid of the Conv3d's in the network? Can we rewrite them to be Conv2D in some manner? I think I can find a workaround for those other dimension related problems since they are all in the detection head.

09-11 16:31:08.935 26380 26380 E tflite  : Following operations are not supported by GPU delegate:
09-11 16:31:08.935 26380 26380 E tflite  : CONV_3D: Operation is not supported.
09-11 16:31:08.935 26380 26380 E tflite  : GATHER: Only support 1D indices
09-11 16:31:08.935 26380 26380 E tflite  : 
09-11 16:31:08.935 26380 26380 E tflite  : RESHAPE: OP is supported, but tensor type/shape isn't compatible.
09-11 16:31:08.935 26380 26380 E tflite  : SOFTMAX: OP is supported, but tensor type/shape isn't compatible.
09-11 16:31:08.935 26380 26380 E tflite  : TRANSPOSE: OP is supported, but tensor type/shape isn't compatible.
09-11 16:31:08.935 26380 26380 E tflite  : 763 operations will run on the GPU, and the remaining 15 operations will run on the CPU.