Assistance Required with Renderer Updates for Multiple XML Models in Reinforcement Learning

[kankannali]

Dear MuJoCo Support Team,

I am currently engaged in reinforcement learning research using MuJoCo. To develop robust strategies, it is crucial for the robot to train across multiple environment models. For this purpose, I have created several XML models. During the environment reset in my reinforcement learning code, I use self.model = mujoco.MjModel.from_xml_path to load a new environment model. I also utilize self.viewer = mujoco.viewer.launch_passive(self.model, self.data) for visualization, which I update upon reset by calling self.viewer._sim().load(self.model, self.data) to visualize the new environment model.

However, I am encountering a challenge with acquiring RGB-D data from the camera. Initially, I declare self.renderer = mujoco.Renderer(self.model, height=720, width=1280)and fetch RGB-D data using the following code:

python
def get_rgbd_data(self):
    camera_name = 'rgbd_camera'
    camera_id = mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_CAMERA, camera_name)
    self.renderer.update_scene(self.data, camera=camera_id)
    rgb_image = self.renderer.render()
    rgb_image = cv2.cvtColor(rgb_image, cv2.COLOR_RGB2BGR)
    self.renderer.enable_depth_rendering()
    depth_image = self.renderer.render()
    self.renderer.disable_depth_rendering()

Each time I execute a reset, I re-declare self.renderer = mujoco.Renderer(self.model, height=720, width=1280) to attempt to capture RGB-D information from the new environment. However, after resetting, I find that the saved RGB-D data does not reflect the new environment model correctly. It is showed in below. Depth image: RGB image:

Could you please advise on how to properly update or reinitialize the renderer so that it correctly reflects changes in the environment model after each reset? Any guidance or additional steps I should consider would be greatly appreciated.

Thank you for your assistance.

[yuvaltassa]

Try renderer.close() before making a new one.

[kankannali]

Your suggestions were very effective and have significantly helped in advancing my project.