[Bug report] Sometimes Z-axis direction changes

[PARKBONG]

Hi. @Toni-SM

I think there's a bug that changes the Z-axis direction of the cameras.

https://user-images.githubusercontent.com/39259734/193912759-9ff7a789-d118-432c-bbe2-e4b2029c1bc5.mp4

Here is properties

https://user-images.githubusercontent.com/39259734/193916411-d6babff7-1e2c-4020-8beb-dfff257fb20b.mp4

1. Not only Add-ons, It sometimes set Z-axis as up, sometime down when I run my code..
2. I also found that setting a focalLength changes camera frames.

from omni.isaac.examples.base_sample import BaseSample
from omni.isaac.core.utils.types import ArticulationAction
from omni.isaac.core.utils.stage import add_reference_to_stage
from omni.isaac.core.robots import Robot
import numpy as np

class HelloWorld(BaseSample):
    def __init__(self) -> None:
        super().__init__()
        self.prim_path = "/Gripper"
        self.name = "my_gripper"
        self.asset_path = "omniverse://localhost/Users/USERNAME/Assets/Robotiq/gripper.usd"
        return

    def setup_scene(self):
        world = self.get_world()
        world.scene.add_default_ground_plane()
        add_reference_to_stage(usd_path=self.asset_path, prim_path=self.prim_path)
        gripper_robot = world.scene.add(Robot(prim_path=self.prim_path, name=self.name))
        return

    async def setup_post_load(self):
        self._world = self.get_world()
        self._gripper = self._world.scene.get_object(self.name)
        self._gripper_articulation_controller = self._gripper.get_articulation_controller()
        self._world.add_physics_callback("sending_actions", callback_fn=self.send_robot_actions)

        # XR
        self.xr = _openxr.acquire_openxr_interface()
        self.xr.init()
        self.xr.create_instance()
        self.xr.get_system()
        self.xr.create_session()

        stage = omni.usd.get_context().get_stage()
        meters_per_unit = UsdGeom.GetStageMetersPerUnit(stage)
        self.xr.set_meters_per_unit(meters_per_unit)
        # **Error2** After set focalLength, local camera frame & rectification changed
        self.xr.setup_stereo_view(camera_properties = {"focalLength": 30}) 

        # **Error1** When Error occurs, z-axis down, requires rotation = pxr.Gf.Vec3d([-90,0,0]) & flip = None
        self.xr.set_reference_system_pose(position = pxr.Gf.Vec3d([0.1,0.1,1]), rotation = pxr.Gf.Vec3d([90,0,0]))
        self.xr.set_frame_transformations(flip=0)
        self.xr.set_stereo_rectification(y=0) 
        self._world.add_physics_callback("HMD", callback_fn=self.on_simulation_step)
        return

    def send_robot_actions(self, step_size):
        euler_ZYX = sim_orientation.as_euler("ZYX",degrees=False)
        self._gripper_articulation_controller.apply_action(ArticulationAction(joint_positions=np.array([0,0,0,0,0,0])))
                                                                                              joint_efforts=None,
                                                                                              joint_velocities=None)) 
        return

    def on_simulation_step(self, step):
        if self.xr.poll_events() and self.xr.is_session_running():
            self.xr.poll_actions()
            self.xr.render_views(_openxr.XR_REFERENCE_SPACE_TYPE_LOCAL)

    async def setup_pre_reset(self):
        print("press : setup_pre_reset")
        _openxr.release_openxr_interface(self.xr)
        return

SYSTEM

• Ubuntu 20.04
• IsaacSim 2022.01
• SteamVR 1.23.7
• VivePro2 + https://github.com/CertainLach/VivePro2-Linux-Driver

[PARKBONG]

Maybe this is not a bug, but an error in the imu sensor.

[Toni-SM]

Hi @PARKBONG

Sorry for late replay.

For future, I need to update the code to handle different world transformations. Currently the implementation is fixed for z-axis up (default in Isaac Sim). However, other Omniverse application are y-axis up by default

[PARKBONG]

It does not seem to be a matter of reference axis.

If you see the viewport of the movie, viewport2 is z-axis up and viewport3 is z-axis down.

However, I found that if HMD starts up upright, these problems rarely occur.

Thanks. @Toni-SM