Stereo output for the Oculus Rift

[balrog-kun]

I'm trying to get this to work with my Oculus Rift DK2 goggles. I managed to modify /opt/vc/src/hello_pi/hello_video/video.c.48-mm to rescale the video to 960 x 540 (half the Rift's 1920x1080 display size) and duplicate the output. I've also found a config.txt settings that work for the Rift HDMI timing parameters. My current problem is that if I connect the Rift to the Raspberry Pi 2 B's HDMI some part of the frame processing slows down so much that it can't keep up with the received video, latency grows and then it dies. If I run the same modified hello_video version with a normal 1920x1080 monitor connected, I see the duplicate video and no noticeable delay. I have tried disabling the vsync usleep code but this made no difference. If anyone has an idea to try, please comment, otherwise I'll be updating this issue with progress.

Once this works I'm not sure where to post the hello_video changes as the custom hello_video versions (hello_video.bin.48-mm, etc.) are not in this repository only in the images. Ideally this project would have a script that takes the repository contents + a vanilla Raspbian image and produces an EZ-WifiBroadcast image, does something like this exist? I also wonder if we have a script to run wifibroadcast rx on a laptop and pipe the output to gstreamer? I've seen this done but don't have a script.

TODO: rescale & duplicate the OSD, shouldn't be difficult. TODO: somehow make the HDMI settings and the mono/stereo hello_video selection runtime configurable or even better autodetectable, both seem tricky.

[balrog-kun]

Fixed the main issue with the latency: it was an issue with the config.txt's display_rotate usage. Doing display_rotate=3 would make the decoding equally slow on both the Oculus and a normal monitor. I added rotation inside hello_video instead and it's working great, I've got great video on the DK2, now need to actually flight test it.

Next will be the OSD support. Keeping this issue open to figure out how to integrate the .profile, config.txt and hello_video modifications.

[rodizio1]

Great work. I'll answer more detailed when I have more time. Just one thing before you start working on the osd: I have almost re-written every function of it in the meantime (see the RCGroups thread). Either wait a week or two until I release that with a new image or tell me if you would also be happy with an unfinished version of the 'new' osd, then I'll upload the new sources.

[balrog-kun]

Thanks, I'm in no hurry with the OSD, I don't actually have the serial port wired on the tx yet.

[rodizio1]

Sorry for the late reply, still not done with the OSD. Probably another week. But I can tell it's going to look great when it's done :)

Regarding the OSD changes: The new OSD will have global scaling as well as scaling for every single element, so I think making it draw everything in stereo should be easy, just calculate the position and size offsets and call the element drawing functions twice. For the rotation I'm not sure, there is a openvg rotate function, it's already being used for the artifical horizion. I'm not sure what happens when rotating everything with that function and then the horizon again ("nested" rotation or how one would call that?).

Or maybe you have a better idea, something like rotating and/or duplicating the whole dispmanx layer (not sure if that's possible)?

Regarding auto-detection: I'm not 100% sure, but I think it won't work without manually changing the hdmi settings in config.txt. I have once experimented with calling tvservice once every two seconds in the background to detect monitor changes and set resolutions etc. (to not be dependent on fixed config.txt settings), but this turned out to be unstable.

But maybe that could be used as the "knob"? I.e. have the .profile script read the config.txt, if it sees that the user has uncommented the Oculus or OSVR HDK settings, it will call the OSD and the hello_video bin with "--stereo" and "--rotate" (or similar) commandline parameters.

Regarding the build script: This question came up already a few times, problem is, this would make a lot more work and probably won't run stable, just too many variables with changing kernels, Pi firmware and Raspbian distribution. Main goal of this project is to make something that is useable out-of-the box without linux knowledge and I only have limited time.

Making the sources complete on Github is still on my todo list, however, everything is included in the image. Actually I don't really like it the way it is with Raspbian, better would be using something like buildroot, much leaner, "buildable" and maintainable, but well, time ...

If you want me to include those changes (I'd like to), just attach the files here. The hello_video.bin and osd sources should be enough, I can make the changes to the .profile script.

[rodizio1]

1.6RC6 with the new osd code has been released now, I'll see that I update the Github repos next week.