iamlab-cmu / isaacgym-utils

Wrappers and utilities for Nvidia IsaacGym
Apache License 2.0
90 stars 19 forks source link

Fix cam tforms #19

Closed tabula-rosa closed 3 years ago

tabula-rosa commented 3 years ago

tl;dr: Isaac Gym cameras should all be correct now. Moreover, isaacgym-utils should now work with 1.0rc2.

Previously, there were camera problems with 1.0rc1, primarily from bugs in Isaac Gym. NVIDIA released 1.0rc2 with enough bug fixes to allow for us to stabilize our camera code. (Some capabilities are still buggy, namely set_camera_location.) Some changes were needed for rc2 as some behavior was different. In rc1, the camera imaging axis was -z. In rc2, the camera imaging axis is +x when in UP_AXIS_Z mode and -z when in UP_AXIS_Y mode. This required recalculation of the R_real_to_gym_cam matrix (and resulting quaternion).

The fact that the up axis mode affects this is interesting, as it suggests the rotation math is probably also changing based on this flag. Regardless, I would suggest going forward that we primarily support UP_AXIS_Z mode - not just for cameras, but for general use of isaacgym-utils.

This also fixes an issue by using deep copies when applying the rotation matrix upon camera setup. Previously, the shallow copy could cause successive environments to have the rotation re-applied.

Code is tested by plotting the image from both the fixed camera and attached camera. Note that for both cases, the image is consistent with the drawn transform. Moreover, this is consistent for each environment.

Screenshot from 2021-08-16 04-15-18

This will close #13 (by providing a viable workaround to rc1 users) and partially addresses some issues with #1.

tabula-rosa commented 3 years ago

Also part of code test: examples/franka_point_cloud_fusion.py works now! Screenshot from 2021-08-16 04-53-36