search

hollance / YOLO-CoreML-MPSNNGraph

Tiny YOLO for iOS implemented using CoreML but also using the new MPS graph API.
MIT License
929 stars 251 forks source link

YOLOv3, tiny-YOLOv3, tiny-YOLOv2 weird bounding boxes on Iphone X #39

Open TheXeniL opened 5 years ago

TheXeniL commented 5 years ago

I'm having trouble with recognition of objects on iPhone X (and iPhone 6s as I tested it) using the YOLOv3-CoreML and YOLOv2-Coreml. Not changing anything in the code gives weird results as you can see in the picture below: img_0047

Using the predict() function, it appears that iPhone X recognizes objects like the umbrella and others with 100% confidence and depicts bounding boxes as stripes. Changing confidenceThreshold and iouThreshold doesn't give any effect, as well as changing maxBoundingBoxes to 1-5. Using predictUsingVision does not produce any prediction at all on the screen.

The same problem I get with tiny-YOLOv3, YOLOv2 with custom and original models from Darknet using iPhone X.

However, using the same code on iPhone 6 (not s) produce an absolutely opposite result. It can predict objects successfully (using predict() function) with my custom model and original Darknet models: photo_2018-09-11_16-35-01

I assumed that there is a problem with GPU changes Apple did start with iPhone 6s devices, but I haven't found any information about that. Does anybody have an issue like this? Did anybody try using the code on iPhone X device? I explicitly decided to test the code in the repository without any changes to provide a demonstration, the problem isn't about YOLOv3 or tiny-YOLOv3 or even YOLOv2 - the result is the same for me using any YOLO version.

You can find videos demonstrating the work on iPhone 6 and iPhone X:

iPhone X: https://imgur.com/a/EZOpr1W

iPhone 6: https://imgur.com/a/qQZtJAd

I will appreciate any help and assumptions you can provide, but as far as I tried to solve the problem for two weeks I didn't get the result, I wonder if there a positive cases using the code on iPhone X by other people. I tested out using different kinds of YOLO models but was always using the same code basis provided by hollance.

hollance commented 5 years ago

There was a problem with the Metal compiler in iOS 11.3 on the iPhone X. These problems should be solved by iOS 12 (and maybe by 11.4?). But I don't see how that would cause any problems with this particular project since it doesn't have custom shaders.

What version of iOS are you running on your different phones?

TheXeniL commented 5 years ago

For iPhone 6 I tested both iOS 11.4 and currently testing it on iOS 12 beta. For iPhone X I tested it on 11.3 and 11.4 as well.

I can try installing iOS 12 beta on iPhone X and check if it's going to work any better, still a shame if the problem can be about the iOS version because I need to support iOS 11+ for the project.

johndpope commented 5 years ago

@TheXeniL - does this help - https://github.com/Ma-Dan/YOLOv3-CoreML