search

WongKinYiu / yolov7

Implementation of paper - YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors
GNU General Public License v3.0
13.37k stars 4.22k forks source link

How can we export to tflite model to run inference on android device ? #686

Open SanjayGhanagiri opened 2 years ago

SanjayGhanagiri commented 2 years ago

python3 export.py --weights runs/train/exp5/weights/last.pt --include tflite --img 416 --data custom-dataset.yaml

We can do this on yolov5 models. Is this possible on yolov7 ?

rjshrivastav commented 2 years ago

same question

June1124 commented 2 years ago

same question

mfagerdal commented 2 years ago

I can leave a short comment, as I chase this path as well.

You can use pintos openvino to convert to tflite. However you will need to convert to onnx before, which export.py will do for you. But the process is fairly straightfroward.

But it seems to give some OPs that is currently not supported by tflite - std or nightly packages - Pinto addsome magic that is ahead of the tensorflow people.. :-)

An alternate path that I am testing right now is to use onnx-tensorflowpackage.

However, it seems Input in yolov7/pytorch is NCHW and tensorflow/tflite expects NWHC. Im sure there are a lot of way to manage this but what Im going after is to change the graf to manage that input.

But then you also have to manage NMS, in code (faster if native) or graph, some imagemanipulation as the boundingbox in tensorflow lite is normalize 0..1 and then also will need to quantize it to get reasonable speed.

Note2: Im doing it for a flutter app where I have yolov5s fp32 model take 40-50 Ms inference time on iPhone 11 pro, coreml delegated.

SanjayGhanagiri commented 2 years ago

@mfagerdal Thanks for the sharing a lot of info. Am exactly on the same path as well. Trying to test tflite, coreML, torchscript models on flutter app.

Dont you think conversion from onnx to tflite would reduce the model accuracy ?

mfagerdal commented 2 years ago

@SanjayGhanagiri

Dont you think conversion from onnx to tflite would reduce the model accuracy ? No. Only quantization to fp16 and int will do that in general. uint8 will reduce accuracy but much faster BUT will not get coreml delegate speedup (only fp32 and fp16) pytorch for mobile looks promising but seems slow still. coreml is great but not in flutter - ie there is no package yet that can manage simultaneous tflite native on android and coreml native on ios.

VikasOjha666 commented 2 years ago

Read this blog: https://medium.com/geekculture/converting-yolo-v7-to-tensorflow-lite-for-mobile-deployment-ebc1103e8d1e

Construct705 commented 2 years ago

Does somebody solved the "yolov7/pytorch is NCHW and tensorflow/tflite expects NWHC" problem?

vi-jay-m commented 1 year ago

@Construct705 Did you find the solution?

Construct705 commented 1 year ago

No sorry, i switched back to yolov5

vi-jay-m @.***> schrieb am Fr., 30. Dez. 2022, 06:57:

@Construct705 https://github.com/Construct705 Did you find the solution?

— Reply to this email directly, view it on GitHub https://github.com/WongKinYiu/yolov7/issues/686#issuecomment-1367741288, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3TMSHYEUNFY5JYAOPD2LFLWPZ2TJANCNFSM6AAAAAAQCLYE24 . You are receiving this because you were mentioned.Message ID: @.***>

dineshkandan commented 1 year ago

Did you find the solution? if i converted and run tflite in Android Bounding box get mismatched it not giving correct output

Jumabek commented 1 year ago

what we found is pytorch and tf exports onnx format differently NCHW and tensorflow/tflite expects NWHC.

Maybe solution is to train yolov7 in tf and export into tflie?

VikasOjha666 commented 1 year ago

@Jumabek Doesn't TF support both channel first and channel last format?. The same model can be used as it is in TF. The user will just have to swap the channel axes while running inference. I don't know about Android though how can that be done.