search

googlesamples / mlkit

A collection of sample apps to demonstrate how to use Google's ML Kit APIs on Android and iOS
Apache License 2.0
3.51k stars 2.92k forks source link

MLKit iOS on GPU/NPU or reduce CPU load? #186

Open igorzharii opened 3 years ago

igorzharii commented 3 years ago

Hi,

For the last couple of days I'm trying to figure out running MLKit for Pose detection on an iPhone's GPU or NPU for higher efficiency than the CPU. From what I understand there is no GPU/NPU processing possible for now, is there any ETA for that? Or maybe there is a way of reducing the CPU load without reducing quality? (Using the .fast model in practice doesn't change the CPU load much, and there are no other visible things to adjust) The reason behind my question is that on longer recognizing streaks (30-60+ minutes) the phone gets noticeably hot which could probably be avoided with processing done on the GPU/NPU.

Thanks

ksulaimon commented 3 years ago

Hey igorzharii,

We are currently working on an optimized version of our pose detector that uses GPU processing and are hoping to have a solution for this early next year. We are aware that running longer sessions causes the CPU to heat up etc, and we are hopeful that our move to GPU will help with this.

In the meantime, have you tried only running the pose detector when absolutely necessary? I'm not sure what your use case is, but turning off the pose detector intermittently when you don't directly need the pose input definitely helps with the thermal/performance issues.

igorzharii commented 3 years ago

Yep, actually pausing and resuming processing for now. Thanks for the update, looking forward to it!

DmytroUsenko commented 3 years ago

Hey igorzharii,

We are currently working on an optimized version of our pose detector that uses GPU processing and are hoping to have a solution for this early next year. We are aware that running longer sessions causes the CPU to heat up etc, and we are hopeful that our move to GPU will help with this.

In the meantime, have you tried only running the pose detector when absolutely necessary? I'm not sure what your use case is, but turning off the pose detector intermittently when you don't directly need the pose input definitely helps with the thermal/performance issues.

Hey @ksulaimon . Do you have any updates about this issue?

Throvn commented 2 years ago

Hey, I don't want to pressure, but I couldn't find it in the releasenotes. Is it just not a high priority right now, or can one somehow help?

bcdj commented 2 years ago

Unfortunately due to other more important work, this feature hasn't been prioritized yet.

TelOkan commented 1 year ago

Hi everyone. I suggest to use mediaPipe. It is using gpu to poseDetection. I think so it will help at performance.

linnaok commented 1 year ago

Hi everyone. how to use MLKit on NPU?