Closed A-KVerma closed 2 years ago
In the dev doc, it says "The API is CPU-based and runs in real time on most modern smartphones (20 FPS+) and works well with both still image and live video streams."
Unfortunately, due to the hardware variance, on low end or older devices, it can be slower as you observed.
Here are some performance tips: https://developers.google.com/ml-kit/vision/selfie-segmentation/android#tips_to_improve_performance
I have tried the performance on OnePlus6 device with 480p resolution it is taking around 80-90ms and sometimes the time is going beyond 100ms too, its not consistent when i have checked the peak it is 250ms over a period of 10 minutes.
OnePlus6 is a good device i guess. I have already gone through the performance tips page.
Can you suggest something here to help ?
I don't thave a OnePlus6, but I gave it a try on a Pixel3XL device which was released in 2018 as well, with slightly lower frequency CPU cores than OnePlus6. With the ML Kit vision quickstart app's LivePreviewActivity, it gets 30FPS.
Are your numbers from your own app or the ML Kit vision quickstart app? If the former, could you try the ML Kit app on that device and see if the number is better? If you are running other heavy computation along the detection in your app, it will be slower.
The numbers is from my own app, I will check the performance using mlkit app also and will update here. Their are other computation also happening in the app, is it possible to shift some operations on GPU for mlkit, though i know it runs on purely CPU. Can we do any other optimization with mlkit ?
As for GPU, we are actively working on GPU acceleration in ML Kit. However, unfortunately, Selfie is not our first candidate. We plan to launch our first GPU acceleration feature soon and will do some improvements based on feedback, then expand it to more features.
Let us know after you try the ML Kit app.
Describe the bug On low end Android device selfie segmentation is running on less than 10 fps, segmenter processing time is mostly greater than 100ms. I am using 360p resolution. Can it be improved ?
To Reproduce Example Steps to reproduce the behavior in sample app:
If applicable, please include
code snippet
andsample input(image, video, text, etc)
Expected behavior Since it says real time, so i am expecting it to run with greater than 30fps
SDK Info:
Smartphone: Tested it on Redmi Note5, VivoY1 devices, samsung galaxy tab A7
Development Environment: (For Android issue feel free to skip this section)