search

fabio914 / RealityMixer

Mixed Reality app for iOS
GNU General Public License v2.0
783 stars 63 forks source link

Head being cut off #56

Open ThatVRGirl opened 3 years ago

ThatVRGirl commented 3 years ago

Hey there, Been using this for about 2 months now. my main issue is that my head is being cut off randomly throughout the capture. Sometimes is happens rarely, and others it happens multiple times. Are there any ways to fix this or tips? I’ve tried multiple angles which sometimes helps, and tried keeping the room i’m in pretty plain. Hanging up a green screen sheet also tends to help. But that defeats the purpose of not needing a green screen. :( I’ve got a video here for example. ( I tried attaching a video but don’t think it uploaded)

So you can see an example on my YouTube here at 1:29 where my head is cut off.

https://youtu.be/har-4fBjnvU

Fyi: I’m using an iPhoneXR. & an oculus quest 2

ReyoVR commented 3 years ago

Hello, I also experience the same issue. It occurs rarely with my head. But I also wear a pair of black wrist braces/wrist support to protect my wrists against strains and with gripping the controllers. The video quite often cut my hands off (for about 20-30% of the duration, my hands are invisible). I also notice that if I wear some elaborate costumes, my whole body will disappear at times. I believe it is the limitation of the ARKit itself failing to differentiate between the foreground and background. It happens more often when the color contrast is great.

Same as you've mentioned. I find that it helps if the camera is more leveled and is filming at eye level, instead of being angled (e.g. very high up looking down). A plain white background seems to be best for tracking in my case. So as reducing movement. I also find that 1.5-2m seems to be the best distance for tracking. Although without wide angle lens, it can only capture half of my body.

I'm using iphone XS max. This is my channel: https://www.youtube.com/channel/UCI7EBsstks95Z4H6domv7mQ

fabio914 commented 3 years ago

Are there any ways to fix this or tips?

I've noticed that the "person segmentation" doesn't work as well when you're close to the edges of the video, but this is not the case in your video. I had also noticed that my head sometimes disappears when my hands and the controllers are close to my face, and it appears that this is what happened in this case.

I believe it is the limitation of the ARKit itself failing to differentiate between the foreground and background.

Unfortunately there's not much I can do to improve this, this is what ARKit provides us and perhaps newer versions of ARKit and iOS will give better results.