Problems with import of omni.isaac.orbit_envs.utils.load_default_env_cfg

[wredsen]

Describe the bug

When directly trying to import: from omni.isaac.orbit_envs.utils import load_default_env_cfg

I am facing the error: ModuleNotFoundError: No module named 'omni.isaac.core'

Steps to reproduce

Just run the following lines of code within your orbit installation:

import gym

import omni.isaac.orbit_envs  # noqa: F401
from omni.isaac.orbit_envs.utils import load_default_env_cfg

cfg = load_default_env_cfg("Isaac-Cartpole-v0")

# create base environment
env = gym.make("Isaac-Cartpole-v0", cfg=cfg, headless=True)

from omni.isaac.orbit_envs.utils.wrappers.sb3 import Sb3VecEnvWrapper
# https://isaac-orbit.github.io/orbit/source/setup/installation.html
env = Sb3VecEnvWrapper(env)

A hacky workaround which enables this import is to create a dummy VecEnvBase before:

from omni.isaac.gym.vec_env import VecEnvBase
dummyEnv = VecEnvBase(headless=True, enable_livestream=False)

System Info

Describe the characteristic of your environment:

• Commit: 2851bb33f455df40cb57e22a33627610020dfd46
• Isaac Sim Version: 2022.2.1
• OS: Ubuntu 22.04
• GPU: [e.g. RTX 4080]
• CUDA: 12.0
• GPU Driver: 525.125.06

Checklist

• [x] I have checked that there is no similar issue in the repo (required)
• [x] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Best regards, Konstantin

[Mayankm96]

Hi, because of how Omniverse works, you need to first run the simulator to be able to load extensions. For more information, check here: https://isaac-orbit.github.io/orbit/source/tutorials/00_empty.html#the-code-explained

[wredsen]

Thanks for pointing this out!

I am trying to do parallel training with Stable Baselines3, that's how I came to orbit:

"""Launch Isaac Sim Simulator first."""
import argparse

from omni.isaac.kit import SimulationApp

# add argparse arguments
parser = argparse.ArgumentParser("Welcome to Orbit: Omniverse Robotics Environments!")
parser.add_argument("--headless", action="store_true", default=False, help="Force display off at all times.")
args_cli = parser.parse_args()

# launch omniverse app
config = {"headless": args_cli.headless}
simulation_app = SimulationApp(config)

# create isaac environment
import gym

import omni.isaac.orbit_envs  # noqa: F401
from omni.isaac.orbit_envs.utils import load_default_env_cfg

# create base environment
cfg = load_default_env_cfg("Isaac-Cartpole-v0")
cfg["env"]["num_envs"] = 128
env = gym.make("Isaac-Cartpole-v0", cfg=cfg, headless=True)

from omni.isaac.orbit_envs.utils.wrappers.sb3 import Sb3VecEnvWrapper
# https://isaac-orbit.github.io/orbit/source/setup/installation.html
env = Sb3VecEnvWrapper(env)

# import stable baselines
from stable_baselines3 import SAC

# create agent from stable baselines
model = SAC(
    "MlpPolicy",
    env,
    batch_size=1000,
    learning_rate=0.001,
    gamma=0.99,
    device="cuda:0",
    ent_coef='auto',
    verbose=1,
    tensorboard_log="./cartpole_tensorboard",
)
model.learn(total_timesteps=1000000)
model.save("sac_cartpole_sb3paral")

env.close()

I am wondering if there is another possibility to create the env besides gym.make (for example task based see the IsaacSim tutorial) before passing it to the Sb3VecEnvWrapper.

Also when it comes to evaluating the inference, how would you recommend to visualize the agent interacting with the environment? Is it possible to connect via the streaming client in some way?

[Mayankm96]

Sorry for the late reply.

Yes it is possible to make the environment without going through gym.make. As an example, you can check the script:

https://github.com/NVIDIA-Omniverse/Orbit/blob/devel/source/standalone/tutorials/03_envs/run_cartpole_rl_env.py#L39-L41

On the devel, it is possible to connect to the streaming client. We tried to optimize the workflows to move more in the cloud deployment direction. You can check the documentation here:

https://github.com/NVIDIA-Omniverse/Orbit/blob/devel/docs/source/tutorials/00_sim/launch_app.rst#the-code-execution

[Mayankm96]

Please feel free to re-open the issue if the problem is unresolved.