iOS iDetection Speed Table

[glenn-jocher]

🚀 Feature

We'd like to create an iDetection Speed Table for each year's iPhone release, going back as far as iDetection will run, which may be around iPhone 5 or 6. Crowd-sourcing the table will help show real-world performance of the models on various hardware.

If you have an iOS device, you can contribute by downloading iDetection, testing your speeds, and then uploading your results here. We try to test near 100% battery at room temperature (about 25°C), and record the speed after about 10 seconds of inference for each model.

Please help by uploading your results! Thank you!!

iDetection v7.8 Inference Speeds

	Year	ASIC -process	ANE (TOPS)		YOLOv5s (ms)	YOLOv5m (ms)	YOLOv5l (ms)	YOLOv5x (ms)
iPhone 6	2014	A8-20nm	-	90	180	350	500
iPhone 6s	2015	A9-16nm	-	148	216	304	475
iPhone 7	2016	A10-16nm	-	94.7	140.7	216.7	289.8
iPhone 8/X	2017	A11-10nm	0.6	-	-	-	-
iPhone XR/XS	2018	A12-7nm	5.0	22.3	25.8	43.2	57.7
iPhone 11	2019	A13-7nm	6.0	17.4	21.3	27.8	41.2
iPhone 12	2020	A14-5nm	11.0	14.3	16.5	21.0	28.8
iPhone 13	2021	A15-5nm	15.8

CoreML models exported as FP8 320x192 with release v3.1 Measured with iDetection v7.8 at 100% battery at 25°C. Average speed after 10 seconds recorded.

[NanoCode012]

Hi Glenn, I got an old 7Plus. I think the results will definitely be lower than a new one.

5s - 10.4 FPS - 94.7 ms 5m - 7.1 FPS - 140.7 ms 5l - 4.6 FPS - 216.7 ms 5x - 3.7 FPS - 289.8 ms

Started from 100%. Default Params.

[glenn-jocher]

@NanoCode012 great, thanks bud! I've added your data points to the table.

[glenn-jocher]

Here are the results of a recent run for iDetection v7.8 YOLOv5s iPhone 11 for 20 minutes, starting from 94% battery at room temperature. The x axis is frame count. About 36000 frames were processed over 1200s (20 minutes), equal to about 30 FPS. You can see initial speed of 17 ms matches table numbers for this device, with thermal throttling starting at about 10 minutes, with iOS automatically reducing processing capacity to about 60% original, resulting in inference times of 27 ms. I don't track memory or thermal state, I'll try to add them in the future. But according to the below charts, everything seems to be working well. It's also possible that longer runs will result in more throttling, so I'll try a longer run to 60 minutes in the future also.

[glenn-jocher]

Additional profiling information for iDetection v7.8 YOLOv5s iPhone 11 run. Most importantly, RAM usage is steady over time and very low at about 100 MB out of 4 GB.

RAM Usage

CPU Usage

Energy Impact

[glenn-jocher]

Updated results for iDetection v7.8 YOLOv5s and YOLOv5m on iPhone 11 for 60 minutes of continuous usage (about 120,000 images processed each). Plots include system RAM usage over time, indicating very small app memory usage (only 70 MB for YOLOv5s and 80 MB for YOLOv5m! :). Conclusion: iPhone 11 is able to run YOLOv5s/m steady state at or near realtime 30 FPS, even after thermal throttling begins around 15 minutes into usage. Results are excellent. Plan to reproduce same test on iPhone 12 soon.

EDIT: Raw data uploaded below for above plots. frames_iPhone11_YOLOv5s.txt frames_iPhone11_YOLOv5m.txt

[glenn-jocher]

iPhone 11 vs iPhone 12 results for iDetection v7.8 YOLOv5s 60 minutes of continuous usage (>100,000 images processed each). Plots include system RAM usage over time, indicating very small app memory usage. Conclusion: iPhone 11 is able to run YOLOv5s/m steady state at or near realtime 30 FPS, even after thermal throttling begins around 15 minutes into usage. iPhone 12 is able to run YOLOv5s with zero throttling over 1 full hour of usage.

EDIT: Raw data uploaded below for above plots. frames_iPhone11_YOLOv5s.txt frames_iPhone12_YOLOv5s.txt

[christli7]

Do you also need iPhone XR? V5s 22 ms V5m 25 ms V5l 44 ms V5x 57 ms

[glenn-jocher]

@christli7 thanks for the iPhone XR numbers! Since the XR and the XS share the same A12 ASICs we can rely on the XS numbers, which are almost exactly the same same as your XR numbers. I will update the table to note that the speeds for XS apply to XR.

[abfleishman]

old iPhone SE (2016?) V5s 148ms V5m 216ms V5l 304ms V5x 475ms

[glenn-jocher]

@abfleishman thanks for the profiling results! iPhone SE is on the A9 chipset, I will update the table with your values!

[jeannotes]

the results are confusing. you said that: V5s 148ms V5m 216ms V5l 304ms V5x 475ms thease numbers are times per image? but you also mentioned that, iPhone 12 2020 A14-5nm 11.0 four models runing time:14.3 16.5 21.0 28.8 what's the the meaning

[glenn-jocher]

@jeannotes table times are correct. Units are milliseconds per image in our iOS YOLOv5 app. See https://github.com/ultralytics/yolov5/issues/1276#issue-735508557

[jeannotes]

@glenn-jocher so how do you do that? any model compression?can you give us some toturials?

[fanatic-studio]

Where can I get the source code of iDetection, to test my own model

[glenn-jocher]

@view-team we have not open-sourced the iDetection source code, however soon you'll be able to preview your own custom models in iOS and Android using the Ultralytics HUB. Please visit https://ultralytics.com/hub to learn more.

[dubrovin-sudo]

CoreML models exported as FP8 320x192

@glenn-jocher How to export a model with a resolution 320x192 if only one number needs to be specified in the argparse?

For example, the default image size of 640 means 640x640: !python utils/benchmarks.py --weights yolov5n.pt --imgsz 640 --device 0

[glenn-jocher]

@dubrovin-sudo --imgsz 320 192

[Marcof64]

Have tested YOLOv5 (m and x) and YOLOv8 (m and l) on iphone 13 Mini (A17, NPU rated 15 TOPS) vs. Iphone 15 Pro (A17 NPU rated 35 TOPS).

Suprisingly I get NO decrese in inference time!!! How is it possibile?

The 15 Pro NPU has 2.3 the TOPS of the 13 mini NPU!

YOLOv5m: 15 ms on both iPhone 13 and 15 Pro YOLOv5x: 21 ms on both iPhone 13 and 15 Pro

YOLOv8m: 14 ms on both iPhone 13 and 15 Pro YOLOv8l: 21 ms on both iPHone 13 and 15 Pro

Iphone 13 mini screnshots:

Iphone 15 Pro screenshots

[glenn-jocher]

@Marcof64 Interesting results! There can be various factors contributing to this, including software optimizations and other bottlenecks. We'll investigate the performance differences further. Thank you for sharing your findings!

[Marcof64]

Hi Glen,

Good to hear from you and congratulations with the Ultralytics growth and YOLO VISION 22.

Let me know if I can be of any help with further test! I have both phones with me and access to more.

Side question: in the new “UltralyticsApp”, “640 imagesize” refers to the training resolution or the resolution being fed to YOLO for inference? What is the latter?

Thanks!!!

Marco

From: Glenn Jocher @.> Sent: Friday, November 17, 2023 3:21 PM To: ultralytics/yolov5 @.> Cc: Marcof64 @.>; Mention @.> Subject: Re: [ultralytics/yolov5] iOS iDetection Speed Table (#1276)

@Marcof64 https://github.com/Marcof64 Interesting results! There can be various factors contributing to this, including software optimizations and other bottlenecks. We'll investigate the performance differences further. Thank you for sharing your findings!

— Reply to this email directly, view it on GitHub https://github.com/ultralytics/yolov5/issues/1276#issuecomment-1816514254 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AY5FOLHVABST2WMSSO5XCKDYE5XC5AVCNFSM4TJBDZM2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRGY2TCNBSGU2A . You are receiving this because you were mentioned. https://github.com/notifications/beacon/AY5FOLDRXWNV2KPGFMNAF6TYE5XC5A5CNFSM4TJBDZM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGONRC45TQ.gif Message ID: @. @.> >

[glenn-jocher]

@Marcof64 Thank you for your kind words and for offering further testing! Your support is greatly appreciated. In the new UltralyticsApp, the "640 imagesize" refers to the resolution being fed to YOLO for inference. Thank you for your attention to detail and willingness to contribute!