facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.76k stars 1.31k forks source link

Render Densepose example: IndexError: tensors used as indices must be long, byte or bool tensors #1784

Open vandoitruong opened 6 months ago

vandoitruong commented 6 months ago

Dear Pytorch3D,

I am trying to run the render_densepose.ipynb, but the error occured at below cell:

# create our verts_uv values
verts_uv = torch.cat([U_norm[None],V_norm[None]], dim=2) # (1, 7829, 2)
# There are 6890 xyz vertex coordinates but 7829 vertex uv coordinates. 
# This is because the same vertex can be shared by multiple faces where each face may correspond to a different body part.  
# Therefore when initializing the Meshes class,
# we need to map each of the vertices referenced by the DensePose faces (in verts, which is the "All_vertices" field)
# to the correct xyz coordinate in the SMPL template mesh.
v_template_extended = v_template[verts -1][None] # (1, 7829, 3)

IndexError Traceback (most recent call last) Cell In[82], line 8 2 verts_uv = torch.cat([U_norm[None],V_norm[None]], dim=2) # (1, 7829, 2) 3 # There are 6890 xyz vertex coordinates but 7829 vertex uv coordinates. 4 # This is because the same vertex can be shared by multiple faces where each face may correspond to a different body part.
5 # Therefore when initializing the Meshes class, 6 # we need to map each of the vertices referenced by the DensePose faces (in verts, which is the "All_vertices" field) 7 # to the correct xyz coordinate in the SMPL template mesh. ----> 8 v_template_extended = v_template[verts -1][None] # (1, 7829, 3)

IndexError: tensors used as indices must be long, byte or bool tensors

Then I modified it to remove this error: v_template_extended = v_template[verts.long() -1][None] # (1, 7829, 3)

However, it have same problem in renderer(mesh) line:

images = renderer(mesh)
plt.figure(figsize=(10, 10))
plt.imshow(images[0, ..., :3].cpu().numpy())
plt.axis("off");

IndexError Traceback (most recent call last) Cell In[80], line 1 ----> 1 images = renderer(mesh) 2 plt.figure(figsize=(10, 10)) 3 plt.imshow(images[0, ..., :3].cpu().numpy())

File c:\Users\ABC\anaconda3\envs\torch110\lib\site-packages\torch\nn\modules\module.py:1102, in Module._call_impl(self, *input, *kwargs) 1098 # If we don't have any hooks, we want to skip the rest of the logic in 1099 # this function, and just call forward. 1100 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks 1101 or _global_forward_hooks or _global_forward_pre_hooks): -> 1102 return forward_call(input, **kwargs) 1103 # Do not call functions when jit is used 1104 full_backward_hooks, non_full_backward_hooks = [], []

File c:\Users\ABC\anaconda3\envs\torch110\lib\site-packages\pytorch3d\renderer\mesh\renderer.py:62, in MeshRenderer.forward(self, meshes_world, kwargs) 49 """ 50 Render a batch of images from a batch of meshes by rasterizing and then 51 shading. (...) 59 For this set rasterizer.raster_settings.clip_barycentric_coords=True 60 """ 61 fragments = self.rasterizer(meshes_world, kwargs) ---> 62 images = self.shader(fragments, meshes_world, **kwargs) ... 991 ] 992 faces_verts_uvs = torch.cat(packing_list) 993 texture_maps = self.maps_padded()

IndexError: tensors used as indices must be long, byte or bool tensors

How can I fix that? Thank you so much!

bottler commented 2 months ago

This works for me.

verts = torch.from_numpy((ALP_UV["All_vertices"]).astype(int)).squeeze().to(device)

should be creating a tensor of dtype int64. What type is it giving you? Can you fix it?