Support Google Cardboard mobile-based Virtual Reality via on-mobile sensors (IOS and Android)?

[sixman9]

Hi, Would it be at all possible to patch Minko's Oculus routines with mobile-sensor-based display orientation data, from IOS and/or Android, ala Google Cardboard (say by accessing the Gyroscope etc.)?

I'm _less_ than a newbie to Minko, however, could Minko's existing Oculus rendering routines, in NativeOculus.cpp for instance and possible new, mobile-centric, implementations of code like your Oculus-libovr (linux-osx/windows), do the trick, on mobile?

I'm really impressed by your comprehensive Oculus-Rift support, however, I'd really like to target a less tech-savvy games market/price-point, and introduce more people to the VR experience.

Thanks

[JMLX42]

Hello,

thank you for your feedback and your documented question.

VR is made of two things:

• stereoscopic rendering properly projected in order to compensate for the lens distortion;
• head orientation (/position) tracking with sensors.

In the case of the Oculus VR:

• rendering is performed 100% by Minko but based on some pre-computed mesh provided by libovr,
• sensors are 100% handled by libovr, which provides the camera orientation/position in order to update the camera Transform component.

In order to support the Google Cardboard (or any mobile VR headset as I understand it):

• stereoscopic rendering has to be done pretty much the same way as for the Oculus VR, but with a different distortion mesh;
• sensors are to be handled using the device onboard sensors obviously.

Someone ported the Google Cardboard SDK for iOS: https://github.com/equinox2k/CardboardVR-iOS That's what I would use as an inspiration to make it work into Minko.

In order to abstract VR and make the code 100% portable, I would probably create an HMDDevice interface that provides the view params, the distortion mesh and the camera orientation/position. Then another HDMVRCamera class, that should be a PerspectiveCamera drop-in replacement and that would behave exactly the same in any case and be parametrized only by the values provided by a specified HMDDevice derived object (OculusHMDDevice, MobileHDMDevice...).

We would probably refactor our existing WebVR code (see http://oculus-rex.com) to fit this scheme too.

We will probably work on this before April. Are you interested in helping out?

Regards,

[sixman9]

@promethe42, although my post was meant to 'lead the charge' on Minko Cardboard integration, I personally have little to no experience in VR techniques and algorithms and I don't currently consider myself to be a competent C/C++ developer (I'm just a lowly salaried Java dev, really). That said, I wouldn't wish to disappoint anyone by promising something I might not be able to deliver (for which I apologise for in advance) .

My main aim in asking for the integration was in order to 'translate' some VR game ideas I have, in the most expediente manner (and cross-platform). I'm aware of several other 3D frameworks which have their corresponding VR-related issues (i.e. lack of an implementation, pricing, etc.), so for now at least, I'd probably like to focus my attention where it's easiest for me to develop my ideas, quickly, however, I really do wish the Minko project every success with a potential Cardboard integration.

My humble apologies

Update: 22/01/15

@promethe42, just reading back your outline for Cardboard integration, and having also found various source decompiles and ports of Google's cardboard.jar binary (from Android), I'm guessing that the integration wouldn't be too hard. However, in my case, if I'm pushed to manually integrate the Cardboard experience into a cross-platform framework, I'm tempted to stay in my comfort-zone of Java, otherwise, I'd probably help out here - let's see how I get on (fingers crossed).