Open mp051998 opened 7 months ago
As part of this, I got it working. I modified the tensors and it works, but the inference time is extremely slow.
I tried exporting my yolov5 model to tflite using non-maximum suppression. However, on starting up the interpreter then I get this error:
Select TensorFlow op(s), included in the given model, is(are) not supported by this interpreter.
I'm not sure what to update, how to update and where all. The documentation step has quite a few steps which are confusing me. Can anyone help?
For faster tensor operation on images you can use this library. Its discontinued but still works(just fork it in your personal github) with images 3.0.0. I have done a speed comparison here https://github.com/tensorflow/flutter-tflite/issues/171 which shows almost 8X speed up on CPU(could be higher on GPU)
Regarding your second question, this happens when you go from torch to onnx to tflite. I am not sure why, but some operations become unsupported while converting.
So please find a tensorflow model, convert it to tflite and use it, you shouldn't face this issue.
For faster tensor operation on images you can use this library. Its discontinued but still works(just fork it in your personal github) with images 3.0.0. I have done a speed comparison here https://github.com/tensorflow/flutter-tflite/issues/171 which shows almost 8X speed up on CPU(could be higher on GPU)
Regarding your second question, this happens when you go from torch to onnx to tflite. I am not sure why, but some operations become unsupported while converting.
So please find a tensorflow model, convert it to tflite and use it, you shouldn't face this issue.
Thanks for the reply @saurabhkumar8112 . Since MediaPipe might take a while I think this might be my best shot. Have to see how that goes! Any idea how to have these inferences make use of the GPU? Does it use the GPU by default?
@mp051998 Thinking of what you mentioned, I recognized a same issue was mentioned in #167. To be honest, try another model that can be converted to tflite. If you still want to use the same model, would you mind trying to containerize the model and then deploy it to the app in another way instead of using tflite and flutter-tflite?
had the same issue. How do you solve this pr ?
Hi,
I've been trying to enable object detection on flutter using a custom trained yolov5n model. I have been using the https://pub.dev/packages/flutter_vision package. It currently has support only for android, not sure when (if ever) iOS support will be added for it. Anyhow, the inference was painfully slow and not really usable for my use case. I have been trying to integrate my app with the flutter-tflite library directly, however I am having issues with setting up the output tensors I think. I have been trying to tweak the 'live_object_detection_ssd_mobilenet' example to get it work with my model but it doesn't work. I understand that this is because both models are very different, I just wish there was an example with YOLOv5 so I'd know how to do it.
This is the netron properties info of my model, can anyone guide me on how to get this working? If there is an example, I could follow that as well.
Also, will the inference time be fast enough as well?
I went through several threads and pages online before posting that issue so as to not waste anyone's time. Hope I can finally put these struggles behind me now.
Thanks!
I'm also attaching the snippet from the example where inference happens for reference.