Environment Support for Real time Go2 Balancing

[ShaoshuSu]

Hi there,

Thanks a lot for making this repo, which is very interesting and helpful.

I was really impressed by the Real-time Go2 Balancing example you showcased in the README. However, when I attempted to replicate it in a similar environment, the balancing failed, as shown in the following screencast:

Screencast from 08-28-2024 10:58:54 PM.webm

Do you have any idea why it might be failing this way? Could it be that my environment is too challenging? If possible, could you share the USD file you used for testing the balancing performance?

Thanks a lot in advance!

[abizovnuralem]

Hello! What version of IsaacSim are you using? It should be 4.0

[ShaoshuSu]

Hello! What version of IsaacSim are you using? It should be 4.0

I see. As I have installed 4.1.0, is there a quick way to switch these two? Btw, I think it seems hard to find the install package of 4.0.0 right now.

[abizovnuralem]

@ShaoshuSu Yes, you can go to the Omniverse launcher and switch the version there. Use "exchange" tab for it

[ShaoshuSu]

Hi @abizovnuralem, thanks for your instruction. It does helps.

I think in order to switch to 4.0.0, besides switching in the launcher, we probably need to switch to isaaclab 4.0.0 in the conda environment as well based on a modification of this instruction.

pip install isaacsim==4.0.0.0 isaacsim-extscache-kit-sdk==4.0.0.0 isaacsim-extscache-physics==4.0.0.0 --extra-index-url https://pypi.nvidia.com

However, in my case, I think there is not too much difference between 4.1.0 and 4.0.0. The Go2 still struggles with climbing stairs.

If it is possible to take some of your time sharing the code and environment settings to reproduce that balancing result, I would be most grateful!

For testing purposes or for anyone who might need it, I can share a rough terrain USD file I generated based on procedural_terrain.py following this instruction.

[abizovnuralem]

All the code is shared here on this repo, it should work as on the screenshots and videos... btw I will check your USD and tell you the results

[abizovnuralem]

Also, I can see that the scene import is not correct in your case, from the video, you have collision mesh with this USD, which is not accurate, maybe that info will help

[abizovnuralem]

In order to use the 4.0 version you also need:

1. Use this specific IsaacLab repo version: https://github.com/isaac-sim/IsaacLab/releases/tag/v1.0.0
2. Inside the root folder of IsaacLab repo execute ln -s /home/{PUT YOUR USERNAME HERE}/.local/share/ov/pkg/isaac-sim-4.0.0 _isaac_sim
3. ./isaaclab.sh --conda
4. sudo apt install cmake build-essential
5. ./isaaclab.sh --install
6. Add there to your .bashrc file, it will push the conda env to use 4.0 version export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-4.0.0" export ISAACSIM_PYTHON_EXE="${ISAACSIM_PATH}/python.sh"
7. Verify a final installation with "python source/standalone/tutorials/00_sim/create_empty.py" For the first time, you need to wait. It will be very slow.
8. You need to check that you have "Isaac Sim Python 4.0.0 - New Stage*" on the top of the window.

[abizovnuralem]

@ShaoshuSu To use "non-flat" ground with balancing, you need to consider how to create an environment with proper collisions. Simply copying the USD file from the train.py example will not work because, for the robot, it will treat it as a wall or another physical object and ignore it. To simulate ground collisions correctly, you can analyze the train.py file and see how the IsaacLab team implemented the technique that forces the robot to respond to ground-like collisions.

[abizovnuralem]

Also, I have noticed the "speed drop" after the IsaacSim 4.1 release, and going back to 4.0 is not helping to fix this issue, the dog is slow. On readme videos, you can see that it is much faster than your demo video here. Need to investigate the problem

[ShaoshuSu]

Hi @abizovnuralem,

Thanks for your reply. I tried your detailed instructions for installing 4.0.0, I think it does help to speed up the simulation.

But it still fails to climb the stairs. I think it does not always fail, sometimes it can climb 1-2 stages and fail later on. I will try the scale method you mentioned. And maybe try to retrain following the pipeline to handle collision as you suggest.

Thank a lot again for the detailed information and suggestions!

[abizovnuralem]

Ok, I have fixed the speed issue. Sorry for the "bad" advice. You need to use IsaacSim 2023.1.1 version instead, here is the full instruction:

1. Use this specific IsaacLab repo version: https://github.com/isaac-sim/IsaacLab/releases/tag/v0.3.1
2. execute in cmd export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-2023.1.1" export ISAACSIM_PYTHON_EXE="${ISAACSIM_PATH}/python.sh" and also put it inside .bashrc file
3. Inside the root folder of Orbit repo (https://github.com/isaac-sim/IsaacLab/releases/tag/v0.3.1) execute ln -s ${ISAACSIM_PATH} _isaac_sim
4. ./orbit.sh --conda
5. conda activate orbit
6. sudo apt install cmake build-essential
7. ./orbit.sh --install
8. ./orbit.sh --extra rsl_rl
9. verify the installation using "python source/standalone/tutorials/00_sim/create_empty.py"
10. You need to check that you have "Isaac Sim Python 2023.1.1 - New Stage*" on the top of the window.
11. go to the current repo project (this repo) and then checkout to the previous version that supports orbit, not IsaacLav 4.0 using this command "git checkout 3a6738cdff70fd2ff8802fd99745887843d1a9f7"
12. execute ./run_sim.sh

After doing all these, can you please give me your feedback does it help to boost the speed? If yes, I will update the current master branch to support orbit instead of IsaacLab 4.0 and then I can help you with custom collisions. Thanks for your help, @ShaoshuSu !

[abizovnuralem]

Thanks for your reply. I tried your detailed instructions for installing 4.0.0, I think it does help to speed up the simulation.

@ShaoshuSu, Can you please provide a video demo of the simulation speed with 4.0? I need to compare it with mine, thanks! Also, can you tell me what balancing task you want to solve? We can create a simple example of it. Thanks!

[ShaoshuSu]

@abizovnuralem The materials and information you provided are very helpful!!!

I think you are right! issacsim is getting slower as its version increases. Here are I attached 3 screen recordings for the versions 2023.1.1, 4.0.0, and 4.1.0. As can be observed, 2023.1.1 is much faster than the upgraded versions. And 4.1.0 is the slowest.

Version 2023.1.1 2023.1.1_version.webm

Version 4.0.0 4.0.0_version.webm

Version 4.1.0 4.1.0_version.webm

Here are just a few more things I would like to suggest based on my experience to follow the instructions:

1. ./orbit.sh --conda might take a very long time to resolve the environment. In my case, I just manually install the required packages in _isaac_sim/environment.yml.
2. When I run git checkout 3a6738c, it fails due to not finding the submodules and not finding the previous commit. Just for anyone who wants to clone this repo, it is suggested to use git clone https://github.com/abizovnuralem/go2_omniverse/ --recurse-submodules -j8 to make sure your clone is complete.

Thanks a lot for your suggestion about the sample example for the rough terrain! What we want to do is a balancing task in an outdoor scenario with complex rough terrain and stairs, which is similar to what produced in procedural_terrain.py. I tried to generate the terrain with the code. I think it works well with the code by modifying def setup_custom_env (attached below). But not sure it doesn't work well when I was trying to save it to a USD file and load it again in the simulation. Btw, as omni.isaac.lab.terrains is not supported in 2023.1.1, I haven't fully successfully tested rough terrain in that version of issacsim.

Also, if you could share some insights about what needs to be noticed when saving/transferring and loading existing environment USD into the isaacsim simulation, we would be really grateful!

from omni.isaac.lab.terrains import FlatPatchSamplingCfg, TerrainImporter, TerrainImporterCfg
from omni.isaac.lab.terrains.config.rough import ROUGH_TERRAINS_CFG  # isort:skip

def setup_custom_env():
    try:
        if (args_cli.custom_env == "rough_terrain"):

            # Parse terrain generation
            terrain_gen_cfg = ROUGH_TERRAINS_CFG.replace(curriculum=False, color_scheme="none") # type: ignore

            # Handler for terrains importing
            terrain_importer_cfg = TerrainImporterCfg(
                num_envs=2048,
                env_spacing=3.0,
                prim_path="/World/ground",
                max_init_terrain_level=None,
                terrain_type="generator",
                terrain_generator=terrain_gen_cfg,
                debug_vis=True,
            )
            # Create terrain importer
            terrain_importer = TerrainImporter(terrain_importer_cfg)
        else:
            cfg_scene = sim_utils.UsdFileCfg(usd_path=f"./envs/{args_cli.custom_env}.usd")
            cfg_scene.func(f"/World/{args_cli.custom_env}", cfg_scene, translation=(0.0, 0.0, 0.0))

            print(f"[INFO] Custom environment loaded: {args_cli.custom_env}")

    except Exception as e:
        print(e)
        print("Error loading custom environment. You should download custom envs folder from: https://drive.google.com/drive/folders/1vVGuO1KIX1K6mD6mBHDZGm9nk2vaRyj3?usp=sharing")

Thank you so much once again for your prompt response and very helpful instructions!!!

[ShaoshuSu]

Just a follow-up, I think orbit seems to support the rough terrain generation as well. Just modify the code to make it work. But the dog seems still struggling with climbing stairs. But anyway, the trained PPO policy looks to work well in other terrains.

https://github.com/user-attachments/assets/9640c9d5-4110-411f-b591-edc250048a74

[abizovnuralem]

@ShaoshuSu Good to hear, I also tried your USD but I have an error saying it does not have a default prim path. Can you check it?

[abizovnuralem]

BTW, you can modify custom_rl_env.py. To use non-plane terrain_type you should use TerrainImporterCFG with the following params:

terrain = TerrainImporterCfg( prim_path="/World/ground", terrain_type="generator", terrain_generator=ROUGH_TERRAINS_CFG, physics_material=sim_utils.RigidBodyMaterialCfg( friction_combine_mode="multiply", restitution_combine_mode="multiply", static_friction=1.0, dynamic_friction=1.0, ), debug_vis=False, )

Replace it with line 69, where were import a standard plane terrain.

[abizovnuralem]

For that situations, I already have prepared a terrain_cfg.py inside this repo with some example settings for rough terrain, so you can import from there from terrain_cfg import ROUGH_TERRAINS_CFG

[abizovnuralem]

This is an example of the config from terrain_cfg.py. After some research inside a code, you can easily understand the trick of this "wrapper" and create your own environments with non-planar surfaces, where you can test your custom-trained RL algorithms.

Screencast from 09-01-2024 10:38:34 AM.webm

[abizovnuralem]

@ShaoshuSu Just updated the master branch and put the rough terrain example inside it, you can check it, thanks!

[ShaoshuSu]

Thank you so much! That solves my problems.