VrHeadModel.rotation() not giving correct values in VR mode

[moroneyk]

Description

Bug When polling the rotation using VrHeadModel.rotation() values are correct until the user enters VR mode. Once in VR mode the VrHeadModel.rotation() value returned is alway the last value before entering VR mode, i.e. it no longer seems to be updated once the user is in VR.

I am using this to display a menu panel directly on front of the user when they look down below a certain angle, hence the need to know the rotation of the head vertically and horizontally to calculate when and where to display the menu. This still works fine in a 2d setting and previously worked in VR, but no longer seems to. Has VrHeadModel been deprecated? If so what is the current best practice for polling the head rotation?

Expected behavior

VrHeadModel.rotation() continues to update correctly when the user enter VR mode.

Actual behavior

When the user enters VR mode the value returned from VrHeadModel.rotation() is alway the last value before entering VR mode, i.e. it no longer seems to be updated once the user is in VR.

Reproduction

Poll the value of VrHeadModel.rotation() before and after enter VR mode. Once in VR mode the VrHeadModel.rotation() value is no longer updated correctly. Sample code for reproducing the issue is given below. The values logged to the console are no longer updated correctly when the user enters VR mode.

Additional Information

• 'react-360' package version: react-360@1.0.0
• 'react-360-web' package version: react-360-web@1.0.2
• Operating System: Android / iOS / Windows (Oculus Rift)
• Browser: Chrome 68.0.3440.91 / Safari / Firefox (Oculus)
• VR Device: Cardboard / Oculus / iOS

UPDATE : r360.compositor.camera / onReceivedHeadMatrix also does not update in VR

Adding an event listener using RCTDeviceEventEmitter also stops reporting values once a user enters VR. Also the onEnterVR event does not seem to be fired when the user enters VR, adding a listener for the event never never intercepts it.

Polling the camera from r360.compositor.getCamera also seems to stop updating the Matrix objects when the user enters VR.

    RCTDeviceEventEmitter.addListener('onReceivedHeadMatrix', e => {
    // this stops reporting values when the user enters VR
    console.log('Event type', e)
    });

    RCTDeviceEventEmitter.addListener('onEnterVR', () => {
        // this event is never intercepted when the user hits the enter VR button 
        console.log('entered VR');
    });

[moroneyk]

Here is sample code to reproduce the issue:

import React from 'react';
import {
  AppRegistry,
  StyleSheet,
  Text,
  View,
  VrHeadModel,
} from 'react-360';

export default class Hello360 extends React.Component {

  componentDidMount() {
    this.timer = setInterval(() => { this.tick() }, 300);
  }

  tick() {
    const myRotation = VrHeadModel.rotation();
    // Upon entering VR these values never change
    console.log("rotation - ", myRotation);
  }

  render() {
    return (
      <View style={styles.panel}>
        <View style={styles.greetingBox}>
          <Text style={styles.greeting}>
            Welcome to React 360
          </Text>
        </View>
      </View>
    );
  }
};

const styles = StyleSheet.create({
  panel: {
    // Fill the entire surface
    width: 1000,
    height: 600,
    backgroundColor: 'rgba(255, 255, 255, 0.4)',
    justifyContent: 'center',
    alignItems: 'center',
  },
  greetingBox: {
    padding: 20,
    backgroundColor: '#000000',
    borderColor: '#639dda',
    borderWidth: 2,
  },
  greeting: {
    fontSize: 30,
  },
});

AppRegistry.registerComponent('Hello360', () => Hello360);

A video showing the issue can be found here:

VrHeadModel.rotation() Issue

[moroneyk]

If anyone else runs into the same issue, it looks like the correct solution is to do something like:

  // Create a r360 instance
  const r360 = new ReactInstance(bundle, parent, {
    // Add custom options here
    fullScreen: true,
    frame: () => {
      const cameraQuat = r360.getCameraQuaternion();
      // Do whatever calculations/checks are necessary here and then fire an event to trigger the action
      // to be performed based on the camera angles

      if (someCondition == true) {
        r360.runtime.context.callFunction('RCTDeviceEventEmitter', 'emit', [
          'myEvent',
          arg1,
          arg2,
          arg3
        ]); 
      }
    },
    cursorVisibility: "visible",
    ...options,
  });

the r360.getCameraQuaternion() is updated correctly in VR and non-VR modes, and from this all angles and rotations can be calculated.

@andrewimm : I haven't closed this issue as I am unsure if it is intentional that VrHeadModel is not updated in VR mode.

[larrylin28]

@moroneyk
I think the issue is VrHeadModel is not updating with correct camera position as r360.getCameraQuaternion does in legacy ReactVR. We are not supporting VrHeadModel in new React360 runtime.

[moroneyk]

Ok, thanks for the info. I will remove all usage of VrHeadModel from my project.

[kmcrawford]

@moroneyk how do you calculate the angle from getCameraQuaternion since we can't get it from VrHeadModel.rotation()?

[moroneyk]

Hi,

You can calculate the camera angle in the client.js file.

function init(bundle, parent, options = {}) {

    ...

  // Vector pointing out from camera
  const cameraDirection = [0, 0, -1];
  var cameraVerticalDeg = 0;
  var cameraHorizontalDeg = 0;

  ...

  // Create a r360 instance
  const r360 = new ReactInstance(bundle, parent, {
    // Add custom options here
    fullScreen: true,
    frame: () => {
      frameNumber++;
      const cameraQuat = r360.getCameraQuaternion();
      cameraDirection[0] = 0;
      cameraDirection[1] = 0;
      cameraDirection[2] = -1;
      // cameraDirection will point out from the view of the camera,
      // we can use it to compute surface angles
      VRMath.rotateByQuaternion(cameraDirection, cameraQuat);
      const cx = cameraDirection[0];
      const cy = cameraDirection[1];
      const cz = cameraDirection[2];

      const vertAngleRad = Math.asin(cy / Math.sqrt(cx * cx + cy * cy + cz * cz));
      const vertAngleDeg = vertAngleRad * 180 / Math.PI;
      const horizontalAngleRad = Math.atan2(cx, -cz);
      const horizontalAngleDeg = horizontalAngleRad * 180 / Math.PI;
      cameraVerticalDeg = vertAngleDeg;
      cameraHorizontalDeg = horizontalAngleDeg;

      // You can either just read the cameraVerticalDeg, cameraHorizontalDeg
      // angles directly from other classes, or if you are waiting for a 
      // particular condition to trigger an action you can 
      // Broadcast the camera positions to any listeners
      if (someConditionisTrue) {
        r360.runtime.context.callFunction('RCTDeviceEventEmitter', 'emit', [
          'cameraPositionUpdate',
          vertAngleDeg,
          horizontalAngleDeg,
          0,
        ]);
      }
    },

    cursorVisibility: "visible",
    nativeModules: [
        // Any native modules here
    ],
    ...options,
  });

I hope this helps, let me know if you have any questions,

Kevin

[kmcrawford]

That’s awesome Thanks!