Error when creating mesh

[zuuukiiio]

Description

I'm still experienceing an "PyMeshLabException: Error while creating mesh: Vertex matrix is empty." issue as stated and supposedly fix in #184. This occered when I try both the NGP and Nerf Collab notebook. It gave a "Failed to load optimizer." warning and a "PyMeshLabException: Error while creating mesh: Vertex matrix is empty." Complete error message see below.

[INFO] Cmdline: main.py -O --test --workspace output/HaiOceanJar2 --save_mesh [INFO] Trainer: df | 2023-05-13_06-02-00 | cuda | fp16 | output/HaiOceanJar2 [INFO] #parameters: 12240458 [INFO] Loading latest checkpoint ... [INFO] Latest checkpoint is output/HaiOceanJar2/checkpoints/df_ep0200.pth [INFO] loaded model. [INFO] load at epoch 200, global step 20000 [WARN] Failed to load optimizer. [INFO] loaded scheduler. [INFO] loaded scaler. ==> Start Test, save results to output/HaiOceanJar2/results 99% 99/100 [00:09<00:00, 10.69it/s] ==> Finished Test. 100% 100/100 [00:12<00:00, 8.18it/s] ==> Saving mesh to output/HaiOceanJar2/mesh [INFO] marching cubes thresh: 0 (1.1750820768213352e-24 ~ 61.129676818847656) ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /content/drive/MyDrive/stable-dreamfusion/stable-dreamfusion/main.py:138 in │ │ │ │ 135 │ │ │ if opt.save_mesh: │ │ 136 │ │ │ │ # a special loader for poisson mesh reconstruction, │ │ 137 │ │ │ │ # loader = NeRFDataset(opt, device=device, type='test', H=128, W=128, si │ │ ❱ 138 │ │ │ │ trainer.save_mesh() │ │ 139 │ │ │ 140 │ else: │ │ 141 │ │ │ │ /content/drive/MyDrive/stable-dreamfusion/stable-dreamfusion/nerf/utils.py:506 in save_mesh │ │ │ │ 503 │ │ os.makedirs(save_path, exist_ok=True) │ │ 504 │ │ │ │ 505 │ │ if loader is None: # mcubes │ │ ❱ 506 │ │ │ self.model.export_mesh(save_path, resolution=self.opt.mcubes_resolution, dec │ │ 507 │ │ else: # poisson (TODO: not working currently...) │ │ 508 │ │ │ points, normals = self.generate_point_cloud(loader) │ │ 509 │ │ │ self.model.export_mesh(save_path, points=points, normals=normals, decimate_t │ │ │ │ /usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py:115 in decorate_context │ │ │ │ 112 │ @functools.wraps(func) │ │ 113 │ def decorate_context(*args, *kwargs): │ │ 114 │ │ with ctx_factory(): │ │ ❱ 115 │ │ │ return func(args, **kwargs) │ │ 116 │ │ │ 117 │ return decorate_context │ │ 118 │ │ │ │ /content/drive/MyDrive/stable-dreamfusion/stable-dreamfusion/nerf/renderer.py:192 in export_mesh │ │ │ │ 189 │ │ # clean │ │ 190 │ │ vertices = vertices.astype(np.float32) │ │ 191 │ │ triangles = triangles.astype(np.int32) │ │ ❱ 192 │ │ vertices, triangles = clean_mesh(vertices, triangles) │ │ 193 │ │ │ │ 194 │ │ # decimation │ │ 195 │ │ if decimate_target > 0 and triangles.shape[0] > decimate_target: │ │ │ │ /content/drive/MyDrive/stable-dreamfusion/stable-dreamfusion/meshutils.py:82 in clean_mesh │ │ │ │ 79 │ _ori_vert_shape = verts.shape │ │ 80 │ _ori_face_shape = faces.shape │ │ 81 │ │ │ ❱ 82 │ m = pml.Mesh(verts, faces) │ │ 83 │ ms = pml.MeshSet() │ │ 84 │ ms.add_mesh(m, 'mesh') # will copy! │ │ 85 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ PyMeshLabException: Error while creating mesh: Vertex matrix is empty.

Steps to Reproduce

Use Collab to generate mesh.

Expected Behavior

Generate mesh.

Environment

Name: torch Version: 2.0.0+cu118

[MathieuTuli]

184 resolved a scenario where something caused mean_density to be zero resulting in no mesh being created at all. Two questions:

1. Note that the current implementation sets density_thresh for mesh mcubes to either self.mean_density or self.density_thresh. If self.mean_density and self.density_thresh are both zero, then you may still get empty matrix results as you are. What is your density_thresh set to?
2. I assume as a sanity check that the validation outputs show something? Like there is a reasonable 3D scene to reconstruct as a mesh?

[zuuukiiio]

Sorry for the late response. I just get back to work.

1. I'm really a noob at this. Where can I set my density_thresh? I'm running everything using collab notbook mounted to google drive, I'll attach my copy of the notebook here.
2. Yes, there are validation outputs, and the rgb video generated looks pretty okay.

Some context: we are trying to use this to generate some imaginary instrument. The prompts we use can be not so typical. Is it possible that the prompts themselves are causing this issue? At the same time, at my most recent the attempt, when trained at 50 epoch, this empty vertex issue persists, while at 100 epoch, it did successfully generate mesh, but the model generated was nothing close to what was in the rgb video. The generated model is mostly just scattered polygons at the corners of what should've been the actual model.