[Bug Report] ViewerCfg origin type = asset_root is broken

KyleM73 commented 1 week ago

Describe the bug

When training with rsl_rl in the default locomotion/velocity environments, if you switch the viewer origin type to track the asset_root the terrain is not visualized when recording videos. see image below. visualization is not affected when the viewer is attached to the world, only when it tracks the robot. I have confirmed this occurs with multiple locomotion tasks.

Steps to reproduce

Reproduce by pasting this into the __post_init__() of any task cfg:

self.viewer.eye = (2.5, 0.5, 0.3) self.viewer.origin_type = "asset_root" self.viewer.asset_name = "robot"

Go1-Flat

Anymal-D-Flat

System Info

Commit: [9ab6b485bad1b57fb93d119907d749d4d2bf289e] (main)
Isaac Sim Version: [4.0]
OS: [Ubuntu 20.04]
GPU: [RTX 3080 12Gb]
CUDA: [12.2]
GPU Driver: [535.183.01]

Checklist

[x] I have checked that there is no similar issue in the repo (required)
[x] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

[ ] The terrain is visualized normally when the viewer tracks the asset_root

Mayankm96 commented 1 week ago

Hi,

Thanks for reporting this issue.

Is the camera at an angle where the terrain is perpendicular to the camera plane?
Does it also happen for the rough terrain?

KyleM73 commented 1 week ago

1) no, the camera eye is at (2.5, 0.5, 0.3). I tried other angles (eg 3.5, 3.5, 3.5) as well and it did not change anything (example in the image below)

2) just tried it with the rough terrain envs- with the same angle as before (2.5, 0.5, 0.3) it looks like you can see the outline of the pyramid steps, but with the angle (3.5, 3.5, 3.5) you can clearly see the ground, but the color is incorrect

Mayankm96 commented 6 days ago

I am unable to reproduce this issue. I added the following to the post init call in the LocomotionVelocityRoughEnvCfg:

# adds tracking of the robot
self.viewer.eye = (2.5, 0.5, 0.3)
self.viewer.origin_type = "asset_root"
self.viewer.asset_name = "robot"

Here's what I see:

My configuration is as follows:

GPU: NV A6000
Driver: 535.183.01

KyleM73 commented 6 days ago

Are you able to visualize the terrain on flat ground as well? I am able to visualize the terrain as you were for rough terrains, but not for the default flat terrain (None generator, terrain set to ‘plane’, etc.)

Mayankm96 commented 6 days ago

When I ran the flat terrain example, that worked for me.

KyleM73 commented 5 days ago

Hmmm I'm not sure what's going on then. I tried pulling from main, rebuilding the container and retesting but I'm getting the same result- I can visualize the rough terrains fine but when tracking the asset_root the terrain is invisible for flat terrains... What else could be affecting this? same rsl_rl version? Same train.py options? I'm testing using the default env cfgs in isaac.lab_tasks/*

Note that I'm not doing any of this with the GUI, these are all coming from saved wandb videos, everything I do is in headless mode since I can't do x11 forwarding to my MacBook.

Mayankm96 commented 3 days ago

I haven't tested it within a container. Also I was running the random_agent.py script instead of a trained policy.

Will have to check with the above and see if I can reproduce the issue.

isaac-sim / IsaacLab