search

SirLynix / obs-kinect

OBS Plugin to use a Kinect (all models supported) in OBS (and setup a virtual green screen based on depth and/or body detection).
GNU General Public License v2.0
426 stars 29 forks source link

Offset? #29

Closed ShocWave closed 3 years ago

ShocWave commented 3 years ago

I tried playing around with this today, and I noticed that unless I stand 1-2 meters away from the Kinect, there seems to be a very huge offset between the color picture and the depth information. Every seems to be shifted over by 10-20 pixels, leaving some kind of strange looking shadow.

I assume this is because the two cameras are quite far apart from each other, so some distance is needed for it to combine everything correctly.

Is there a possibility that in the future, you could add some manual controls to align the depth and color images? I'm not able to place the camera far away from me due to space constraints.

SirLynix commented 3 years ago

Hello,

Could you please tell me what version of obs-kinect you're using, along with the source setup you have.

Why do you need to combine color and depth streams? The greenscreen effect has depth to color conversion and will handle that for you.

Could you please post an image of what's happening and describe how you're using obs-kinect sources?

Thank you.

ShocWave commented 3 years ago

Sorry, maybe I wasn't very good at explaining it. I've attached an image here: https://imgur.com/y45rT7O Take a look at my hand and thumb, and the side of my head, it seems like the depth sensor thinks my body is shifted to the left.

The further back I move, the less I notice this.

SirLynix commented 3 years ago

What you are experiencing isn't caused by a bug / offset, it's an occlusion shadow.

This is caused by the color and IR camera being one centimer apart (+ the IR emitter and IR camera being one centimer apart too). The only way to counteract this is by having a better filter, I've been experimenting with this but I'm lacking some skills.

Could you try using "Body" instead of "Depth" as filter type? This will use Microsoft body detection instead of raw depth values which may improve the result.

ShocWave commented 3 years ago

Yeah, I'm aware this is not really a bug, and caused by the sensors being apart from each other. Would it be possible to include some manual adjustments to shift the IR and Color data so that it can be adjusted to work with closer distances?

I tried Body, Depth, and Body/Depth and it didn't change the result much. The only thing that works is standing further away.

I think the original Kinect had a snap-on lens accessory to help with this, but I couldn't find anything similar for Kinect v2.

SirLynix commented 3 years ago

Well, yes it's possible to let the user offset the depth image but I really don't think it will help. As I said it's not an offset problem but a missing information problem. If you offset the depth image to the left side you're gonna be cropped on the right one.

SirLynix commented 3 years ago

Closed for no reply. Unfortunately there's not much I can do here.