search

hollance / YOLO-CoreML-MPSNNGraph

Tiny YOLO for iOS implemented using CoreML but also using the new MPS graph API.
MIT License
935 stars 252 forks source link

Performance #56

Open 1omarsaid opened 5 years ago

1omarsaid commented 5 years ago

Its no doubt that there are a lot of computations that it needs to do to detect the objects, however having said that is there a way that we can better the performance? I know we can reduce the frame rate and the amount of times it detects but are there other things we can do to better the performance?

hollance commented 5 years ago

Use an iPhone XS or better. :-)

On older devices, YOLO doesn't really perform very well. The best thing you can do there is switch to something like MobileNet+SSD: https://machinethink.net/faster-neural-networks/

On the XS's Neural Engine, YOLO is actually pretty fast.

So "to better the performance" needs a bit more context: which device will you be running it on?

1omarsaid commented 5 years ago

I just wanted to make sure that when I saw performance, I dont mean the accuracy of the detection but the battery usage and the phone getting hot when the app is running. Is there a way to better that? It will be running on iPhones 7 and up.

hollance commented 5 years ago

On anything from iPhone 7 up to (and including) the X, the best way to get better speed is to not use YOLO but MobileNetV2+SSDLite. (I don't have any recommendations on how to make YOLO faster.)

1omarsaid commented 5 years ago

I think you misunderstand me, I am not looking to make it faster, I want to better the battery consumption.

hollance commented 5 years ago

That's really the same thing. The more efficiently a model runs, the faster it is, and the better the battery usage is. See also https://machinethink.net/blog/how-fast-is-my-model/

1omarsaid commented 5 years ago

What is your advice if I want to setup an object detection application like Tiny YOLO on iOS but have improved battery usage. I went through some of the articles that talked about MobileNetV2+SSDLite, would it run on all devices? and from your experience how much more efficient is it in comparison to YOLO? Thank you again

hollance commented 5 years ago

It is at least 4x faster / more efficient than (tiny) YOLO v2. Only on the XS / XR is YOLO faster because it can take advantage of the Neural Engine better.

1omarsaid commented 5 years ago

How can that be? I ran the MobileNetV2+SSDLite demo app in Xcode and measured the Energy Impact it was very high, and the CPU usage was over 50%. At the same time, I ran the TinyYolo app with some modifications (less frames per second) and was able to get the Energy consumption to be eventually in the low sector with about 20% CPU usage

hollance commented 5 years ago

Did you run both at the same number of FPS? Naturally, if MobileNetV2+SSDLite runs as fast as it can, it will use more energy than TinyYOLO running at a lower frame rate.

If you want to get a rough idea of the energy usage of a neural network, count up the number of memory accesses that are being done by a single pass through the network (see that blog post I linked to). Memory accesses use up a lot more energy than computations, so that would be the thing to focus on.