Can I get the results from a bird's eye view（BEV）?

[missTL]

Can I get the results from a bird's eye view（BEV）?

[ziyc]

Hi @missTL, currently we don't have specific BEV rendering support. We'll consider adding it in future updates. You can simulate BEV by setting a high camera position with a downward viewing angle when rendering novel views by adding a novel view function in this script.

[MMMaverick]

Hello, I have implemented the BEV (Bird's Eye View) interface in camera.py as you suggested, but during actual rendering, the camera moves forward in the shooting direction, causing the view to go below the ground. Do you know how to solve this issue? @ziyc

[MMMaverick]

Here's my code to try on the nuscenes dataset :`def bev_poses_trajectory( dataset_type: str, per_cam_poses: Dict[int, torch.Tensor], original_frames: int, target_frames: int ) -> torch.Tensor:

assert 0 in per_cam_poses.keys()
key_poses = adjust_camera_pose(per_cam_poses)
return interpolate_poses(key_poses, target_frames)

def adjust_camera_pose( per_cam_poses: Dict[int, torch.Tensor] ) -> torch.Tensor: """ Adjust the camera pose:

1. Increase the camera height by 5 meters.

2. Rotate the camera to look down (BEV view).

   Args: per_cam_poses (Dict[int, torch.Tensor]): Dictionary of camera poses.

   Returns: torch.Tensor: Adjusted camera pose of shape (N, 4, 4). """ assert 0 in per_cam_poses.keys(), "Front center camera (ID 0) is required"

   current_pose = per_cam_poses[0]

   translation_matrix = torch.tensor([ [1, 0, 0, 0], # x [0, 1, 0, -5],# z [0, 0, 1, 0], [0, 0, 0, 1] ], dtype=torch.float32).cuda()

   angle = -np.pi / 6
   rotation_matrix = torch.tensor([ [1, 0, 0, 0], [0, np.cos(angle), -np.sin(angle), 0], [0, np.sin(angle), np.cos(angle), 0], [0, 0, 0, 1] ], dtype=torch.float32).cuda()

   transform_matrix = torch.mm(translation_matrix, rotation_matrix)

   adjusted_poses = torch.stack([torch.mm(transform_matrix, current_pose[i]) for i in range(len(current_pose))])

   return adjusted_poses `