Super Interesting!

[dnck]

Hey, this is a really interesting project. I'm dipping my toes into the VR waters with a VR-ready PC and a Oculus Go. I've briefly tested streaming PC Games to the Go via ALVR, Vridge, and Virtual Desktop. VRidge was OK, but the video stream quality was poor. ALVR I did not get to work with recognising my GO on the the same wifi lan. So far, Virtual Desktop had the easiest integration with SteamVR games, and the best quality video stream on the Go, but I only had the one 3Dof Go controller, which had incorrect mappings for the buttons used in the game, which seems to require 6Dof controllers. My question, then, is how to conceptualize the architecture of your modifications. As far as I understand, it goes like this...

1. FreePie acts as interface between controllers and ALVR.
2. ALVR acts as the interface between HMD with controllers and SteamVR.

The purpose of FreePie is to translate the buttons in real time to the game controls. This is sent to ALVR which is sent to ALVR. ALVR then handles the SteamVR api.

Is this correct?

I'm especially interested in setting up a cheap 6dof mod to the GO. I saw some stuff using PSMove controllers, etc. Any tips there?

[AltoRetrato]

Hi!

Your assumptions are correct.

If you use ALVR, try the latest stable (v.2.3.1). I also could not make the beta versions work.

You can see some options for adding 6 DoF to a 3 DoF VR headset in this Reddit thread: Who is planning to play Half Life: Alyx on an Oculus Go?.

I'm sure you can add 6 DoF to Oculus Go, but it can take a considerable amount of time to do it well. And if you don't do it well, lag and jitter and other issues might cause nausea and provide a subpar experience, so unless you are in it for the journey instead of the destination, that is a trip I would not recommend.

To be honest, I thought about it and did a few tests. Most of the USB webcams I've seen have too much lag. A Raspberry Pi with an add-on board camera is relatively cheap and much, much faster (not only much lower lag, but much higher frame rate). There are several computer vision algorithms that you could use with a single camera to track LEDs on the Oculus Go and two controllers, for example, but even if the devices are visible by the camera all the time (meaning you can't turn around), there are still chances of occlusion (e.g., having one controller blocking the LEDs of the headset or of another controller), so you also need an algorithm to fuse the devices' IMU data. In order to have more mobility, you will need more cameras, and the complexity of the system will also grow. But I think this would be a reasonably cheap solution, even though it requires custom programming.

[dnck]

Of course, I'm in it for the journey, but I don't think the travel expenses should be as high as they are!

For starters, two new move controllers plus two cameras comes out to about 120. If the solution doesn't work, I'd just end up with more plastic and metal waste.

It seems like the state of VR is being set back considerably by Facebook. What would be cool is something like what you're talking about - attach an on board usb cam to a pi 4, and run the tracking algo on there, and publish the result on the lan for an interface to steamvr.

[AltoRetrato]

A Raspberry Pi with a camera module can cost less than half of that. And if your solution doesn't work... you can still do a ton of stuff with that hardware! 😁

Please note that I'm not talking about a USB camera (as I wrote before, it would be too slow for such application), but about a camera module. The Raspberry Pi (and other single-board computers) have a 15-pin MIPI camera interface (CSI) connector, so data flows almost directly from the camera to the GPU RAM. You might even start to process part of the image data before all data is transferred.

One idea I had even before the Quest was released was to put markers on the ceiling, and a Raspberry Pi with a camera on top of a Gear VR or Oculus Go, with the camera pointing up. With ideal lighting and Wi-Fi conditions, position could be estimated and sent to the headset or a PC, allowing several users to interact together in the same virtual space with cheap headsets, even over very large areas.

But I partially disagree with your opinion about Facebook: they (and others) are moving the state of the art forward in an incredible way. Just look at the success that is the Quest. If they can release a new version for an even lower price (which might soon be possible), I'm sure it would quickly outsell PSVR. Anything like what we are talking about here is not very consumer friendly...