search

derenlei / Unity_Detection2AR

Localize 2D image object detection in 3D Scene with Yolo in Unity Barracuda and ARFoundation.
https://derenlei.medium.com/object-detection-with-localization-using-unity-barracuda-and-arfoundation-794b4eff02f8
MIT License
222 stars 61 forks source link

Performance Increase #22

Closed KracelKany closed 3 years ago

KracelKany commented 3 years ago

Is there anything in the project which is omittable? I am still trying to fully understand the hierarchy and the project's workflow, but the main issue is with the template is that the FPS becomes incredibly low while detecting something. Would performance increase with custom model and lower number of object to detect?(Like 3-4)

BTW, a big round of applause for this project, it is very useful, you are very skilled.

Also, I got these warnings, what could be the culprit? image

derenlei commented 3 years ago

Hi @KracelKany,

I think the low FPS issue is not due to the warnings, but more depends on your phone. I'm not an Unity expert and used to get some warnings. I guess it's ok as long as all the components are functioning haha.

I tested on a Samsung S10 and iPhone Xs. iPhone is observably faster while S10 is slow but still acceptable. This performance gap looks significant if using tiny Yolo3.

One thing that could help is to do image detection with fewer frames: https://github.com/derenlei/Unity_Detection2AR/blob/main/Assets/Scripts/PhoneARCamera.cs#L126. This method currently feeds all the received frames into the recognition model during detection.

KracelKany commented 3 years ago

Thank you very much, that is a good idea!