metadriverse / metadrive

MetaDrive: Open-source driving simulator
https://metadriverse.github.io/metadrive/
Apache License 2.0
746 stars 107 forks source link

Specifying position/hpr in `perceive` doesn't work when using DepthCamera #635

Open fredyshox opened 7 months ago

fredyshox commented 7 months ago

Trying to follow the example from: https://metadrive-simulator.readthedocs.io/en/latest/obs.html#customization-multiview, but using DepthCamera and it doesn't seem to work. The resulting images are not taken in new positions/orientations.

To work around this I'm using multiple depth cameras and calling BaseCamera.track before the step to update positions, which works, but it would be nice to re-use single cam for this.

Observation code:

class Observation(BaseObservation):
  def __init__(self, config):
    super(Observation, self).__init__(config=None)
    self.rgb = ImageObservation(config, "rgb", True)
    self.depth = ImageObservation(config, "depth", True)

  @property
  def observation_space(self):
    os = {key: getattr(self, key).observation_space for key in ["depth", "rgb"]}
    return gym.spaces.Dict(os)

  def observe(self, vehicle):
    os = {key: getattr(self, key).observe(vehicle.origin, position=(0, -5, 0), hpr=(0,0,0)) for key in ["depth", "rgb"]}
    return os

Image: rgb_depth