search

isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
2.32k stars 957 forks source link

[Question] Selecting objects to render in (or to filter out from) Camera sensors #399

Closed titoirfan closed 5 months ago

titoirfan commented 6 months ago

Question

Hi, thanks for the great work on Orbit.

I am doing vision-based locomotion, so I have front-facing onboard cameras on my robots.

While the robots do not collide with each other during training, they still see each other in their (RGB) cameras. This can clutter the robot's vision, especially if the number of parallel environments is high.

Is there a way to select which objects to (or not to) render in the robot's camera view?

Thanks for your time and support.

ninamortensen commented 6 months ago

Hi there, do you mind sharing your setup? I can't seem to get the basic rendering to work (e.g. https://github.com/NVIDIA-Omniverse/orbit/blob/main/source/standalone/tutorials/04_sensors/add_sensors_on_robot.py). The demo only runs when the camera sensor is commented out, otherwise it throws the following error:

Program aborted due to an unhandled Error: Unable to find target for this triple (no targets are registered)Aborted (core dumped)

titoirfan commented 6 months ago

@ninamortensen I personally did not have any trouble running that demo script.

My setup is:

I also have libllvm11 ver. 1:11.1.0-6 and libllvm15 ver. 1:15.0.7-0ubuntu0.22.04.3 if that helps. I followed the official tutorials for installing Isaac Sim and Orbit (using the Orbit conda environment).

The only trouble I had was when I tried to run the demo in headless mode, for which I figured out I have to use the offscreen_rendering flag.

ninamortensen commented 6 months ago

@ninamortensen I personally did not have any trouble running that demo script.

My setup is:

* Isaac Sim Version: 2023.1.1

* OS: Pop!_OS 22.04

* GPU: RTX 4060

* CUDA: 12.4

* GPU Driver: 550.67

I also have libllvm11 ver. 1:11.1.0-6 and libllvm15 ver. 1:15.0.7-0ubuntu0.22.04.3 if that helps. I followed the official tutorials for installing Isaac Sim and Orbit (using the Orbit conda environment).

The only trouble I had was when I tried to run the demo in headless mode, for which I figured out I have to use the offscreen_rendering flag.

Got it, thanks a lot!

hutslib commented 4 months ago

Question

Hi, thanks for the great work on Orbit.

I am doing vision-based locomotion, so I have front-facing onboard cameras on my robots.

While the robots do not collide with each other during training, they still see each other in their (RGB) cameras. This can clutter the robot's vision, especially if the number of parallel environments is high.

Is there a way to select which objects to (or not to) render in the robot's camera view?

Thanks for your time and support.

Hi, I meet the same problem and is this fixed? @titoirfan @Dhoeller19