facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.54k stars 1.28k forks source link

Seam when rendering triangle close up #1209

Open tobyclh opened 2 years ago

tobyclh commented 2 years ago

If you do not know the root cause of the problem / bug, and wish someone to help you, please post according to this template:

🐛 Bugs / Unexpected behaviors

I am trying to render a simple quad created by two triangles, but there is a seam when rendered from a close distance.

Instructions To Reproduce the Issue:

Please include the following (depending on what the issue is):

  1. Any changes you made (git diff) or code you wrote Minimal working example based on the tutorial
    
    import os
    import sys
    import torch
    import pytorch3d
    import os
    import torch
    import matplotlib.pyplot as plt
    import math

from pytorch3d.io import load_objs_as_meshes, load_obj

Data structures and functions for rendering

from pytorch3d.structures import Meshes from pytorch3d.vis.plotly_vis import AxisArgs, plot_batch_individually, plot_scene from pytorch3d.vis.texture_vis import texturesuv_image_matplotlib from pytorch3d.renderer import ( look_at_view_transform, FoVPerspectiveCameras, PointLights, DirectionalLights, Materials, RasterizationSettings, MeshRenderer, MeshRasterizer,
SoftPhongShader, TexturesUV, TexturesVertex )

add path for demo utils functions

import sys import os sys.path.append(os.path.abspath(''))

Setup

if torch.cuda.is_available(): device = torch.device("cuda:0") torch.cuda.set_device(device) else: device = torch.device("cpu")

obj_filename = 'plane.obj'

mesh = load_objs_as_meshes([obj_filename], device=device) fov = 45 half_fov_rad = (fov / 2) / 180.0 * math.pi height = 0.5 / math.tan(half_fov_rad) R, T = look_at_view_transform(height, 0, 0) cameras = FoVPerspectiveCameras(device=device, R=R, T=T, fov=fov)

raster_settings = RasterizationSettings( image_size=512, blur_radius=0.0, faces_per_pixel=1, )

Place a point light in front of the object. As mentioned above, the front of the cow is facing the

-z direction.

lights = PointLights(device=device, location=[[0.0, 0.0, 1.0]])

Create a Phong renderer by composing a rasterizer and a shader. The textured Phong shader will

interpolate the texture uv coordinates for each vertex, sample from a texture image and

apply the Phong lighting model

renderer = MeshRenderer( rasterizer=MeshRasterizer( cameras=cameras, raster_settings=raster_settings ), shader=SoftPhongShader( device=device, cameras=cameras, lights=lights ) )

images = renderer(mesh)

plt.figure(figsize=(10, 10))

plt.imshow(images[0, ..., :3].cpu().numpy()) plt.show()


plane.obj and plane.mtl

mtllib plane.mtl o Cube v 0.5 0.5 0.0 v -0.5 0.5 0.0 v 0.5 -0.5 0.0 v -0.5 -0.5 0.0 vt 0.0 1.0 vt 1.0 1.0 vt 0.0 0.0 vt 1.0 0.0 usemtl material_1 s off f 4/3/1 1/2/1 2/1/1 f 4/3/1 3/4/1 1/2/1

newmtl material_1 map_Kd banana.jpg

Test colors

Ka 1.000 1.000 1.000 # white Kd 1.000 1.000 1.000 # white Ks 0.000 0.000 0.000 # black Ns 10.0


banana.jpg
![image](https://upload.wikimedia.org/wikipedia/commons/thumb/8/8a/Banana-Single.jpg/2324px-Banana-Single.jpg)
3. The exact command(s) you ran:
python render.py

5. What you observed (including the full logs):
there is a seam between the two triangles.
![image](https://user-images.githubusercontent.com/12501995/170475209-3ce11a4b-2cf3-4100-9f53-c70d1e4c2b03.png)
github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] commented 1 year ago

This issue was closed because it has been stalled for 5 days with no activity.

kjchalup commented 1 year ago

Thanks for reporting and great repro steps!

I checked that the seam disappears if you change to blur_radius=1e-5. Also, there's no blur if you use the new MeshRasterizerOpenGL (though you need pycuda.gl and pyopengl for that!).

Nevertheless, this looks like a bug in MeshRasterizer -- it shouldn't be leaving holes even with blur_radius=0.0. I'll try to look into it when I get some time.