search

hollance / YOLO-CoreML-MPSNNGraph

Tiny YOLO for iOS implemented using CoreML but also using the new MPS graph API.
MIT License
935 stars 252 forks source link

Adding orientation changes - Effect on Bounding boxes #29

Closed vvkv closed 6 years ago

vvkv commented 6 years ago

Thank you for your work on this. I am just working on incorporating orientation changes, so that the the camera feed rotates as the screen is rotated while still bounding objects. Im working with the front camera on my iPad pro and everything works perfectly in portrait mode (as it should). To allow for the landscape orientation I add "Landscape (left of home button)" and "Landscape (right of home button)" in the info.plist. I also change line 50 previewLayer.connection?.videoOrientation = .portrait to previewLayer.connection?.videoOrientation = .landscapeRight and line 66 videoOutput.connection(with: AVMediaType.video)?.videoOrientation = .portrait to videoOutput.connection(with: AVMediaType.video)?.videoOrientation = .landscapeRight in https://github.com/hollance/YOLO-CoreML-MPSNNGraph/blob/master/TinyYOLO-CoreML/TinyYOLO-CoreML/VideoCapture.swift

While this gets the screen rotation working, the bounding boxes become a lot bigger than the objects being shown. How can I incorporate screen rotation while keeping the objects still tightly bounded.

Thank you

hollance commented 6 years ago

It might be necessary to rotate the texture 90 or 270 degrees for this to work. It's been a while since I looked at this code, so I can't tell off the top of my head what is needed. Using Vision it's simply a matter of passing the orientation into the request handler.