[Bug Report] default_camera_config in MujocoEnv seems not working

[Naming-isDifficult]

Describe the bug

I was trying to understand how default_camera_config works via adjusting its values. However, I surprisingly found that no matter how I adjusted its values (at least distance and elevation), it didn't affect the result.

Code example

Import statements:

import gymnasium as gym
import numpy as np
import matplotlib.pyplot as plt

To reproduce the ant-v4 one, install gymnasium-0.29 and run:

distances = [0, 10, 20, 30, 40, 50]
elevations = [0, -10, -20, -30, -40, -50]

results = []

for distance in distances:
    for elevation in elevations:
        gym.envs.mujoco.ant_v4.DEFAULT_CAMERA_CONFIG = \
        {
            "distance" : distance,
            "elevation" : elevation
        }

        env = gym.make('Ant-v4', render_mode='rgb_array')
        env.reset()

        param = env.unwrapped.mujoco_renderer.default_cam_config
        param = (param["distance"], -param["elevation"])

        img = env.render()

        results.append((param, img))

To reproduce the humanoid-v5 one, install gymnasium-1.0.0 and run:

distances = [0, 10, 20, 30, 40, 50]
elevations = [0, -10, -20, -30, -40, -50]

results = []

for distance in distances:
    for elevation in elevations:
        gym.envs.mujoco.humanoid_v5.DEFAULT_CAMERA_CONFIG["distance"] = distance
        gym.envs.mujoco.humanoid_v5.DEFAULT_CAMERA_CONFIG["elevation"] = elevation

        env = gym.make('Humanoid-v5', render_mode='rgb_array')
        env.reset()

        # get distance and elevation from default_cam_config stored in
        # mujoco_renderer to show it does take some effects
        param = env.unwrapped.mujoco_renderer.default_cam_config
        param = (param["distance"], -param["elevation"])

        img = env.render()

        results.append((param, img))

To plot the result, run:

fig, axis = plt.subplots(6, 6)
fig.set_figheight(25)
fig.set_figwidth(25)

for ((i,j), img) in results:
    axis[i//10, j//10].imshow(img)
    axis[i//10, j//10].set_title(f"distance:{i}, elevation:{-j}")

plt.show()

### System info

`gymnasium-0.29` is installed via running `pip install gymnasium`
`gymnasium-1.0.0` is installed via cloning this github repo and running `pip install -e .`

`mujoco` and `mujoco-py` are installed via running `pip install mujoco, mujoco-py`

Other information:

python=3.8 mujoco=3.2.2 mujoco-py=2.1.2.14

OS:
`Ubuntu 22.04 LTS`

### Additional context

_No response_

### Checklist

- [X] I have checked that there is no similar [issue](https://github.com/Farama-Foundation/Gymnasium/issues) in the repo

[Kallinteris-Andreas]

have you tried using the default_camera_config argument? (it is included in v5 of the environments)

[Naming-isDifficult]

have you tried using the default_camera_config argument? (it is included in v5 of the environments)

Yes, I did. It doesn't help either. You could modify the code to be

distances = [0, 10, 20, 30, 40, 50]
elevations = [0, -10, -20, -30, -40, -50]

results = []

for distance in distances:
    for elevation in elevations:
        camera_config = {
            "distance": distance,
            "elevation": elevation
        }

        env = gym.make('Humanoid-v5',
                       render_mode='rgb_array',
                       default_camera_config=camera_config
                      )
        env.reset()

        # get distance and elevation from default_cam_config stored in
        # mujoco_renderer to show it does take some effects
        param = env.unwrapped.mujoco_renderer.default_cam_config
        param = (param["distance"], -param["elevation"])

        img = env.render()

        results.append((param, img))

And you will get the same result.

[Naming-isDifficult]

I read the code thoroughly again and I'm suspecting the camera setting itself.

During initialization of MujucoRenderer, it uses the following code to get camera_id when user does not define either camera_id or camera_name:

if camera_id is None:
        self.camera_id = mujoco.mj_name2id(
                self.model,
                mujoco.mjtObj.mjOBJ_CAMERA,
                camera_name,
        )

The result of this code is the MujocoRenderer.camera_id will be set to be 0, which is the same as MjvCamera.fixedcamid. In other words, most likely the camera used by default is the fixed camera, and according to Mujoco domumentation

mjCAMERA_FIXED This refers to a camera explicitly defined in the model, unlike the free and tracking cameras which only exist in the visualizer and are not defined in the model. The id of the model camera is given by mjvCamera.fixedcamid. This camera is fixed in the sense that the visualizer cannot change its pose or any other parameters. However the simulator computes the camera pose at each time step, and if the camera is attached to a moving body or is in tracking or targeting mode, it will move.

And thus makes default_camera_config not working.

However, I have to admit that I'm not familiar with Mujoco nor Gymnasium. I might be wrong but hopefully this provides some insights to you.

[Kallinteris-Andreas]

is the bug present for render_mode="human"

[Naming-isDifficult]

Update:

Overview

It seems that I might find the problem, partially.

It is caused by MujocoRenderer.camera_id but it is not because MujocoRenderer.camera_id is set to be the same as MjvCamera.fixedcamid ---- MjvCamera.fixedcamid is a property, not a constant.

Here's how things work:

During initialization of MujucoRenderer, the following code will be executed:

no_camera_specified = camera_name is None and camera_id is None
if no_camera_specified:
    camera_name = "track"

if camera_id is None:
    self.camera_id = mujoco.mj_name2id(
        self.model,
        mujoco.mjtObj.mjOBJ_CAMERA,
        camera_name,
    )

The problem is actually caused by this "track" camera.

According to MujocoDocumentaion:

“track” means that the camera position is at a constant offset from the body in world coordinates, while the camera orientation is constant in world coordinates. These constants are determined by applying forward kinematics in qpos0 and treating the camera as fixed.

In other words, the camera obtained in this way is "fixed," relatively, and will not be affected by default_camera_config.

In order to make default_camera_config working, setting MujocoRenderer.camera_id to be -1 works, at least. This can be achieved by assigning a random name to parameter camera_name during initialization. The following code demonstrates this idea:

distances = [0, 10, 20]
elevations = [0, -10, -20]

results = []

for distance in distances:
    for elevation in elevations:
        camera_config = {
            "distance": distance,
            "elevation": elevation
        }

        env = gym.make('Humanoid-v5',
                       render_mode='rgb_array',
                       default_camera_config=camera_config,
                       camera_name='whatever_you_want_:)'
                      )
        env.reset()

        # get distance and elevation from default_cam_config stored in
        # mujoco_renderer to show it does take some effects
        param = env.unwrapped.mujoco_renderer.default_cam_config
        param = (param["distance"], -param["elevation"])

        img = env.render()

        results.append((param, img))

And the result will be:

Possible fix

Quick fix

A really quick fix is to set MujocoRenderer.camera_id to be -1 by default.

Better fix?

As I've mentioned before, I'm not familiar with Mujoco nor Gymnasium, I'm not sure what the side effect is when using the quick fix. According to the documentation, it seems that modifying qpos also helps, but that is far beyond my knowledge.

Something else

At this time I'm not sure if it is actually a bug. It might just need better documentation, maybe. However, I do find a real bug while testing. In MujucoRenderer.__init__(), after the following code:

if camera_id is None:
    self.camera_id = mujoco.mj_name2id(
        self.model,
        mujoco.mjtObj.mjOBJ_CAMERA,
        camera_name,
    )

There should be an else statement:

if camera_id is None:
    self.camera_id = mujoco.mj_name2id(
        self.model,
        mujoco.mjtObj.mjOBJ_CAMERA,
        camera_name,
    )

else:
    self.camera_id = camera_id

Or you might get AttributeError: 'MujocoRenderer' object has no attribute 'camera_id' while calling MujocoEnv.render()

That's all what I can provide, I believe. Hopefully that helps!

[Naming-isDifficult]

is the bug present for render_mode="human"

I haven't tested with render_mode='human' but I believe that is not a problem in that case. The Mujoco visualizer uses a freecamera by default which can be directly adjusted using your mouse.

mjCAMERA_FREE This is the most commonly used abstract camera. It can be freely moved with the mouse. It has a lookat point, distance to the lookat point, azimuth and elevation; twist around the line of sight is not allowed. The function mjv_moveCamera is a mouse hook for controlling all these camera properties interactively with the mouse. When simulate.cc first starts, it uses the free camera.