search

keijiro / NNCam

Virtual background with Unity Barracuda
Apache License 2.0
249 stars 35 forks source link

Vulkan on Android compatibility issue #6

Closed ROBYER1 closed 3 years ago

ROBYER1 commented 3 years ago

Hey, I saw you have been doing a lot of work with Android/IOS lately and I recently put some work in with these guys on getting Barracuda + Yolo v3 tiny working on mobile with AR.

I was working on getting this running on IOS/Android yesterday, I first hit some issue with the webcamtexture not working by setting the FPS of the camera to something like 30 FPS otherwise it would black screen.

I also had to set the WorkerFactory.Type to Csharpburst for CPU work as GPU isn't supported on Android OpenGLES...

Did you have any other pointers on how to get this working on mobile? I would like to help however I can

keijiro commented 3 years ago

It worked on my Pixel 5 with Vulkan. I hardly think that it's practical for non-Vulkan devices.

keijiro commented 3 years ago

I'm closing this issue now. Please feel free to reopen it for further problems.

ROBYER1 commented 3 years ago

On Android did you make any amends to the scripts in the Room scene? For me I have found the webcamtexture is working behind the scenes, but it is rotated sideways (probably an easy fix anyway) and the room image is on screen as expected.

However the actual segmentation isn't working and I have checked I am using Vulkan. Some errors in the log which I cannot find much info about, not sure if unique to Barracuda or Vulkan:

Error SwappyVk Failed to wait for fence 2

Tested on Android 10 with a Sony Xperia XZ3 if that helps

keijiro commented 3 years ago

It's considered a compatibility issue in Vulkan on Android. Which device are you using?

ROBYER1 commented 3 years ago

Sony Xperia XZ3 + Adreno 630

Seems so although Barracuda docs said Vulkan was the standard for Android

ROBYER1 commented 3 years ago

I also know the results will be slower but I am keen to use CPU on android due to the GPU driver issues encountered, would you strongly advise against that?

There aren't really any other solutions I have seen on Unity for mobile that have body segmentation results as good as this demo, hence my efforts to test and get this working on Android/iOS

keijiro commented 3 years ago

Thanks for the information.

Regarding the SwappyVk issue, could you submit it using the bug reporter? I have no idea about it, and the graphics/Barracuda team should look into it.

ROBYER1 commented 3 years ago

Thanks I will do, I am also going to try with CPU only on Android as mentioned, on IOS works fine on Metal as expected. If I have any other major issues I will respond here but thank you for the fast responses 👍

keijiro commented 3 years ago

I also know the results will be slower but I am keen to use CPU on android due to the GPU driver issues encountered, would you strongly advise against that?

I hardly think that it's practical with the CPU implementation because it's quite slow (maybe less than 1 FPS), but you can try it anyway.

keijiro commented 3 years ago

I'm closing this issue now. Please feel free to reopen it for further problems.