facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.7k stars 1.3k forks source link

How can I align rendered image to be in the center? #1789

Closed cmh1027 closed 4 months ago

cmh1027 commented 5 months ago

I'm following the tutorial in here, and it is so far so good except that the image is not rendered in the center.

def render(point_cloud, elev=0, azim=0, image_size=512, radius=0.003, ppp=10):
    R, T = look_at_view_transform(1, elev, azim)
    cameras = FoVOrthographicCameras(device=device, R=R, T=T, znear=0.01)
    raster_settings = PointsRasterizationSettings(
        image_size=image_size, 
        radius = radius,
        points_per_pixel = ppp
    )
    renderer = PointsRenderer(
        rasterizer=PointsRasterizer(cameras=cameras, raster_settings=raster_settings),
        compositor=AlphaCompositor()
    )
    images = renderer(point_cloud)
    plt.figure(figsize=(10, 10))
    plt.imshow(images[0, ..., :3].cpu().numpy())
    plt.axis("off")
pcd = o3d.io.read_point_cloud("pcd.ply")
verts = torch.from_numpy(np.array(pcd.points)).float().to(device)
rgb = torch.from_numpy(np.array(pcd.colors)).float().to(device)
point_cloud = Pointclouds(points=[verts], features=[rgb])
render(point_cloud, elev=2, azim=2, image_size=(253, 381), radius=0.02)

This is my code, and I generate point cloud (pcd.ply) with an image and its pseudo-depth from monodepth estimator. image And the rendered result it shown as follows. image Definitely, not only the result is zoomed in, but also it is not center-aligned (black empty space at the bottom). Is there any post-processing step for point cloud to get correct result? Points in pcd has the mean values of [ -0.0723, 0.2070, -14.5213].

bottler commented 4 months ago

I don't think centre-alignment or the point cloud is the problem here. I think the black part at the bottom of the image is is caused by z-clipping. Adjusting znear may fix it. In general you need to play with the camera settings to get the "zoom" you actually want.