IndexError occurs when passing the `Meshes` with the last one being an empty mesh into `SubdivideMeshes`

[pisceskkk]

🐛 Bugs / Unexpected behaviors

When passing meshes with the last one being an empty mesh into SubdivideMeshes().subdivide_heterogenerous(), out-of-bounds error occurs at https://github.com/facebookresearch/pytorch3d/blob/47d5dc88247035b35ca3cfce159565f92d8fbb75/pytorch3d/ops/subdivide_meshes.py#L383 But it's okay if the last mesh is not empty, regardless of whether the other meshes are empty or not.

Instructions To Reproduce the Issue:

pytorch3d version: pytorch3d==0.7.2 Codes to reproduce:

import torch
from pytorch3d.ops import SubdivideMeshes
from pytorch3d.structures import Meshes

verts_list = [[[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0]], []]
faces_list = [[[0, 1, 2], [0, 2, 3]], []]
verts_list = [torch.tensor(verts, dtype=torch.float64) for verts in verts_list]
face_list = [torch.tensor(faces, dtype=torch.long) for faces in faces_list]
meshes = Meshes(verts=verts_list, faces=face_list)
subdivided_meshes = SubdivideMeshes()(meshes)

outputs:

Traceback (most recent call last):
  File "test_bug.py", line 10, in <module>
    subdivided_meshes = SubdivideMeshes()(meshes)
  File "/home/XXX/miniconda3/envs/XXX/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/XXX/miniconda3/envs/XXX/lib/python3.8/site-packages/pytorch3d/ops/subdivide_meshes.py", line 176, in forward
    return self.subdivide_heterogenerous(meshes, feats)
  File "/home/XXX/miniconda3/envs/XXX/lib/python3.8/site-packages/pytorch3d/ops/subdivide_meshes.py", line 264, in subdivide_heterogenerous
    verts_sort_idx = _create_verts_index(
  File "/home/XXX/miniconda3/envs/XXX/lib/python3.8/site-packages/pytorch3d/ops/subdivide_meshes.py", line 384, in _create_verts_index
    idx_diffs[v_to_e_idx] += v_to_e_offset
IndexError: index 9 is out of bounds for dimension 0 with size 9

Naive Solution

And there are some potential out-of-bounds errors. https://github.com/facebookresearch/pytorch3d/blob/47d5dc88247035b35ca3cfce159565f92d8fbb75/pytorch3d/ops/subdivide_meshes.py#L383-L384 https://github.com/facebookresearch/pytorch3d/blob/47d5dc88247035b35ca3cfce159565f92d8fbb75/pytorch3d/ops/subdivide_meshes.py#L449-L452

I tried to eliminate potential out-of-bounds issues by implementing simple boundary conditions. I'm not sure if it's correct, but it works for me.

diff --git a/pytorch3d/ops/subdivide_meshes.py b/pytorch3d/ops/subdivide_meshes.py
index 8620430..d3eee44 100644
--- a/pytorch3d/ops/subdivide_meshes.py
+++ b/pytorch3d/ops/subdivide_meshes.py
@@ -380,7 +380,13 @@ def _create_verts_index(verts_per_mesh, edges_per_mesh, device=None):

     # Add one new vertex per edge.
     idx_diffs = torch.ones(V + E, device=device, dtype=torch.int64)  # (36,)
+    if len(v_to_e_idx) != 0 and v_to_e_idx[-1] == len(idx_diffs):
+        v_to_e_idx = v_to_e_idx[:-1]
+        v_to_e_offset = v_to_e_offset[:-1]
     idx_diffs[v_to_e_idx] += v_to_e_offset
+    if len(e_to_v_idx) != 0 and e_to_v_idx[-1] == len(idx_diffs):
+        e_to_v_idx = e_to_v_idx[:-1]
+        e_to_v_offset = e_to_v_offset[:-1]
     idx_diffs[e_to_v_idx] += e_to_v_offset

     # e.g.
@@ -446,6 +452,12 @@ def _create_faces_index(faces_per_mesh: torch.Tensor, device=None):
     # pyre-fixme[6]: For 1st param expected `Union[List[int], Size,
     #  typing.Tuple[int, ...]]` but got `Tensor`.
     idx_diffs = torch.ones(4 * F, device=device, dtype=torch.int64)
+    if len(switch1_idx != 0) and switch1_idx[-1] == len(idx_diffs):
+        switch1_idx = switch1_idx[:-1]
+        switch2_idx = switch2_idx[:-1]
+        switch3_idx = switch3_idx[:-1]
+        switch4_idx = switch4_idx[:-1]
+        switch123_offset = switch123_offset[:-1]
     idx_diffs[switch1_idx] += switch123_offset
     idx_diffs[switch2_idx] += switch123_offset
     idx_diffs[switch3_idx] += switch123_offset

[bottler]

Thanks for this report! I think the recent commit is a cleaner solution.