[BUG] Vive Pro cam delivers corrupted image then freezes

[elmar-k]

Describe the bug Dual front camera picture of the HTC Vive Pro flashes intermittently in wrong colors until after a few minutes the camera freezes.

To Reproduce Steps to reproduce the behavior: Go to SteamVR > Settings > Camera > Test camera and performance

System Information (please complete the following information): System 1: Distribution: Ubuntu 22.04.1 LTS SteamVR version: 1.25.1 Steam client version: Nov 17 2022, at 02:45:32 Opted into Steam client beta?: No Graphics driver version: Nvidia RTX2080 515.86.01 Gist for SteamVR System Information: https://gist.github.com/elmar-k/9424b9f9f9d5d58d0cb9e86ef65be2a3

System 2: Distribution: Ubuntu 22.04.1 LTS SteamVR version: 1.25.2 Steam client version: Dec 15 2022, at 21:27:25 Opted into Steam client beta?: No Graphics driver version: Nvidia GTX980 525.60.11

Screenshots

Additional context Streaming the front cameras with VLC works without problems (see second screenshot), hence the assumption that this is a SteamVR bug. Requesting the camera frame from inside an app with OpenVR with GetVideoStreamFrameBuffer produces the same bug. This bug does not appear in Windows. This bug does not appear with the HTC Vive, only the Vive Pro. We tried USB 2 and USB 3 ports, only USB 3 ports allowed to detect the camera.

Please give us a quick hint whether there are any plans to fix this, or if we need to set up a hack to get the camera stream outside SteamVR.

[Abishjp]

Hey there have you found out any solution for this. I am using windows . Please reply

[elmar-k]

Yes, we found that all cameras (also of the first VIVE from 2016) are unusable via OpenVR in Linux, because the call to GetVideoStreamFrameBuffer takes far too long and totally exceeds the 11ms time budget per frame. We tried to place it in a separate thread, but this caused various crashes. After some more research due to a lack of documentation, we concluded that GetVideoStreamFrameBuffer is not thread-safe and must not be called while another thread is doing e.g. WaitGetPoses. We therefore gave up and are using Video4Linux now, which requires to handle a large bunch of things manually for each headset (e.g. decoding of the possibly interlaced stereo or non-stereo frames, estimating the camera pose at the time the shot was taken to avoid long lags and motion sickness). It's a nightmare, but our VR app would be unusable without camera.

[elmar-k]

P.S.: You write that you are using Windows. But in Windows, I think the Vive Pro camera works, no? We only had to recalibrate the shifts of the two camera frames to get a proper stereo view, since the values returned by OpenVR were not correct.

[Abishjp]

I am using vive wireless with vive pro2 . Sometimes the headset's display blacks out. After that i am facing the same issue that you are facing. I need to restart the whole steamvr setup. I cannot tell the user to do this everytime. I am using passthrough in between my game. This frustrates .

Steamvr shows camera is connected but the image is corrupted as yours.