search

Rectus / openxr-steamvr-passthrough

OpenXR API Layer for SteamVR camera passthrough
MIT License
109 stars 5 forks source link

external cameras? #8

Open gagagu opened 1 year ago

gagagu commented 1 year ago

Hi, i'm using a Pimax 8kx which has no cameras. Is it possible to use two external cameras (small webcams, each for every eye) with your software? I want to see my enviroment in game but blended like a greenscreen. Why? I want to use it with the microsoft flight simulator to see my self made cockpit parts in game. I want to make everything green that i dont want to see in game. Is it possible with your software? Thx

Rectus commented 1 year ago

Unfortunately webcams are not supported. Cameras currently need a SteamVR driver to work, and currently the only ones available are built in to HMD drivers.

I've been thinking about trying to write a SteamVR driver for generic webcams, but I don't have any experience with the driver-side interface, so I'm not sure how hard it would be. The biggest issues seem to be if or how to get an exact timestamp for each frame, and how to align the camera with the HMD tracking space.

gagagu commented 1 year ago

Hi, thx for reply. Do you really need to align the cameras in space? The cameras are fixed added to the headset. You can add a menu to align the images (right, left, top, bottom). Why a SteamVr driver is needed? Isn't it possible to grap the webcam frame from windows? Sadly i only have limited programming knowledge, mostly c#, java, and html for database programming.

Rectus commented 1 year ago

It would be possible to just use the Windows camera API, although it would still be almost as much work. I would rather use the SteamVR system though, since it provides readily available predicted poses for the cameras. It's not necessary for having the cameras attached to the HMD, but I'd rather be able to have static cameras, and moveable cameras attached to trackers as well.

The frame timestamp is the most difficult part, since it has to be very accurate to reproject the image correctly. I don't think there is any good way of getting them though, since most consumer cameras don't seem to be generating them.