search

ltkong218 / FastFlowNet

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation (ICRA 2021)
MIT License
249 stars 41 forks source link

Android deployment #3

Open sieme97 opened 3 years ago

sieme97 commented 3 years ago

Have you tried deploying your model on android?

ltkong218 commented 3 years ago

Currently, I have not deployed my model on Android devices. My experience is that correlation operation can only approach real-time when acclerated by parallel computing devices, such as GPU or NPU, etc. This is different from monocular depth networks, which can perform quickly on CPU. I think my future work will consider this application problem.

sieme97 commented 3 years ago

Can't we use mobile gpu?

ltkong218 commented 3 years ago

Of course we can use. The Android device with mobile gpu is ideal, and is the trend. The deployment takes some engineering work.

sieme97 commented 3 years ago

Thanks for your response. So this means we can't convert the trained models to tflite version directly?

ltkong218 commented 3 years ago

You can load provided pre-trained weights in PyTorch and convert them to the TFLite or ONNX for deployment.

sieme97 commented 3 years ago

How to take care of correlation operation when converting the model to tflite?

ltkong218 commented 3 years ago

I have not tried using TFLite. According to my experience, correlation and warping are not standard operations integrated in existing packages. And you should create custom layers to deploy it, which can be referred to documents and examples of TFLite.

dragen1860 commented 3 years ago

@ltkong218 thanks for your advice. When considering pruned and deployed on mobile phone devices, which model/algorithm would you recommend to use ?

ltkong218 commented 3 years ago

Oh, I have not consider pruning yet, that may be a future work.

zdenkas commented 2 years ago

Hi @sieme97 , @dragen1860, did you manage to run FastFlowNet on Android? I'm also interested in this.