search

hollance / YOLO-CoreML-MPSNNGraph

Tiny YOLO for iOS implemented using CoreML but also using the new MPS graph API.
MIT License
933 stars 252 forks source link

No bounding-box is detected on iphone6 #10

Closed rainmancao closed 5 years ago

rainmancao commented 7 years ago

Hi, hollance, Thanks a lot for providing this great example. I am trying to reproduce on my iphone. This is my procedure below:

  1. I tested the keras model, it produces the bbox as expected
  2. I converted the keras model to coreML and put the coreML model in the project, when I launch the the app on iphone, there is no bbox is detected, then I printed the size of the predictions for each frame, it is 0. The confidence score is just very low around 10e-15, I guess it is filtered by the threshold.

I used your coreML model from github directly as well, the same results. Could you please provide some suggestions.

Thanks

hollance commented 7 years ago

Are there any error messages in the Xcode debug panel?

rainmancao commented 7 years ago

Hollance, no, there is no error message in the Xcode debug panel. I was thinking the preprocessing step might be an issue. For example the RGB channel order, the pixel value normalization. Can you share any insight that I need to pay more attention to when converting the model?

Thank you very much

hollance commented 7 years ago

If my model also doesn't work on your iPhone 6 then it sounds like an issue in CoreML, since the model works fine on the iPad Pro.

Does the MPSNNGraph version work on your iPhone?

One thing to try is to pass the usesCPUOnly option (it might be spelled differently) to CoreML when it is making the prediction.

Also, just to make sure: are you sure your input image actually contains any objects for the model to recognize?

rainmancao commented 7 years ago

Hi, hollance,

I haven't tried MPSNNGraph version. I do test scene with objects covered, for example, cars, bikes, human, etc.

My xcode is xcode 9 beta 5, but I assume beta 4 or beta 5 will not be an issue. I will try enable the usesCPUOnly option and will update my results

oishi89 commented 6 years ago

I got the same issue and I checked the keras model after converting then the model was not working at this point. No bounding-box until you change the confident score threshold = 0.001xxx. Could you check your model ?

hollance commented 6 years ago

@oishi89 Where/how have you checked that the model isn't working?

I have no iPhone 6 to test this on but the code works fine on the 6s, 7, and iPad Pro.

If the model does not work in Keras, then perhaps the inputs you're giving it are in the wrong format.

oishi89 commented 6 years ago

The input is trained model with my own data using tiny-yolo-voc.cfg. I tested for making sure it works properly before conversion to Keras. I ran the yad2k.py with output is a file.h5 at this point I also ran the test_yolo.py then no object was detected. Do you have any idea please help

hollance commented 6 years ago

It's probably something in yad2k.py then. Note that I modified yad2k.py to make it work with Keras 1.2, since at the time coremltools did not support Keras 2 yet. You might have better luck with the official version of YAD2K.

oishi89 commented 6 years ago

I have tried the official version already and both have the same result. I'm very new with Python and Machine learning. If you have time please review the yad2k.py

hollance commented 6 years ago

Unfortunately I don't have the time to debug your model for free. But my services are for hire if you're willing to pay my hourly rate: http://machinethink.net/hire/