hollance / Forge

A neural network toolkit for Metal
MIT License
1.27k stars 172 forks source link

Forge slows down after few hundred frames #7

Closed marseel closed 6 years ago

marseel commented 7 years ago

Hello,

I am building app that requires real time performance. I've run few tests on Inceptionv3 example and here are results: First run: https://pastebin.com/G3ErxAcA Second run: https://pastebin.com/Se0zHifF

For the first ~300 frames GPU execution time starts with 0.07, but later increases to ~0.085. I thought that possible cause is GPU overheating, but right after first run I tried running app again and on the second run results are similar: first few hundreds frames are processed much faster than last ones.

I see that it also depends on fps: https://pastebin.com/cmFkFP5P I use iPad Pro for testing and I set fps to 15. It runs smoothly for few 300-400 frames and then later slows down a lot (even 0.44 time for one or two frames) and than runs faster ~ 0.11 s, but still slower than at the beginning. This experiment is repeatable: it is always really fast -> two frames super slow -> slower than at the beginning.

What causes this? Maybe some problem with resource management in Forge?

In my app execution time increases from 0.12 s to 0.2s per one execution, which makes my app unusable.

Thanks for help in advance :)

hollance commented 7 years ago

Interesting. It is certainly possible there is some resource management issue, but note that running Inception at 15 FPS is going to be problematic anyway. 15 FPS only gives you 0.0667 sec per frame to work with, so you're going to end up dropping frames.

The Forge example apps aren't really optimized yet for real-world use, so it's possible they have weird bugs. However, Forge itself just calls Metal so there isn't much it can do wrong (but I'm sure there are subtle issues).

I would suggest you try the same experiment with this sample project from Apple: https://developer.apple.com/library/content/samplecode/MetalImageRecognition/Introduction/Intro.html

Can you reproduce the same behavior there?