facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.53k stars 1.28k forks source link

Render object without texture file #1598

Open Frq-F opened 1 year ago

Frq-F commented 1 year ago

❓ Questions on how to use PyTorch3D

When studying the official documents "Render a textured mesh", I found that my .obj file only contain v and f message but have no texture information for example: v -7.54758989 -3.60210822 -93.47303373 0.50196078 0.50196078 0.50196078 f 221 1791 1347 how can I render objects that have no texture information? How can I solve this problem? Thank you for your help!

bottler commented 1 year ago

You can use SoftSilhouetteShader: see the fit_textured_mesh tutorial for an example.

Or you can make up a texture - e.g. a TexturesVertex with a constant color - and assign that to your_mesh.textures and render with that .

Frq-F commented 1 year ago

Thank you for your suggestion. I tried to modify the code, but it still did not render successfully,the entire image only appears in purple. Here is my code. Could you please help me check where the problem is? By the way, I only have obj files and no mtl files。

verts, facesidx, = load_obj('FHA7K\FHA7K\FHA7K_lower.obj') faces = faces_idx.verts_idx

verts_rgb = torch.ones_like(verts)[None] # (1, V, 3) textures = TexturesVertex(verts_features=verts_rgb.to(device))

teeth_mesh = Meshes( verts=[verts.to(device)],
faces=[faces.to(device)], textures=textures )

cameras = FoVPerspectiveCameras(device=device) blend_params = BlendParams(sigma=1e-4, gamma=1e-4)

raster_settings = RasterizationSettings( image_size=256, blur_radius=np.log(1. / 1e-4 - 1.) * blend_params.sigma, faces_per_pixel=100, )

silhouette_renderer = MeshRenderer( rasterizer=MeshRasterizer( cameras=cameras, raster_settings=raster_settings ), shader=SoftSilhouetteShader(blend_params=blend_params) )

distance = 3 # distance from camera to the object elevation = 50.0 # angle of elevation in degrees azimuth = 0.0 # No rotation so the camera is positioned on the +Z axis.

R, T = look_at_view_transform(distance, elevation, azimuth, device=device)

silhouette = silhouette_renderer(meshes_world=teeth_mesh, R=R, T=T) silhouette = silhouette.cpu().numpy()

plt.figure(figsize=(10, 10)) plt.imshow(silhouette.squeeze()[..., 3]) # only plot the alpha channel of the RGBA image plt.grid(False) plt.show()

Frq-F commented 1 year ago

Thank you for your suggestion. I attempted to render using the official provided code “Camera position optimization using differentiable rendering” and teapot data, and it was successful. But my data rendering failed. I found that my obj file not only lacks texture information , but also normal information . Is this the reason why I am unable to render successfully? How to solve this problem?

bottler commented 1 year ago

Normals will be calculated if they aren't provided. When you say you can't render successfully, do you mean that you get a blank image? Can you check the object and camera locations are what you expect, eg with plot_batch_individually([mesh, camera]) from pytorch3d.vis.plotly_vis?

FlyienSHaDOw commented 9 months ago

The vertex of your obj file has a dimension of 6. The first three are the xyz coordinate the vertex, which should be sent to "verts". The later three are rgb colors, which should be assigned to verts_rgb. Does this solves your problem?