lukemelas / realfusion

Official code for "RealFusion: 360° Reconstruction of Any Object from a Single Image" (CVPR 2023)
Apache License 2.0
541 stars 38 forks source link

Saving mesh not working #3

Open tydia opened 1 year ago

tydia commented 1 year ago

Description

Hi @lukemelas, thanks for open sourcing your great work! Upon reproducing your examples, cat_statue specifically, I noticed that --save_mesh option does not work as expected on testing time. Here's the output:

[INFO] Trainer: df | 2023-04-10_14-25-15 | cuda | fp16 | outputs/default/2023-04-10--13-49-56--seed-101/ [INFO] num parameters: 1_806_983 [INFO] num parameters w/ grad: 1_806_983 [INFO] Loading latest checkpoint ... [INFO] Latest checkpoint is outputs/default/2023-04-10--13-49-56--seed-101/checkpoints/df.pth [INFO] loaded model. [INFO] load at epoch 50, global step 5000 ==> Start Test, save results to outputs/default/2023-04-10--13-49-56--seed-101/results 100% 100/100 [00:05<00:00, 18.63it/s]rgb opacity depth /home/tongwang/workspace/realfusion/nerf/trainer.py:590: RuntimeWarning: invalid value encountered in cast preds_np = (preds_tensor.detach().cpu().numpy() * 255).astype(np.uint8) normals textureless grid ==> Finished Test. 100% 100/100 [00:06<00:00, 15.85it/s] ==> Saving mesh to outputs/default/2023-04-10--13-49-56--seed-101/mesh ==> Finished saving mesh.

Although the log says it is "==>Saving mesh", but it did not actually save the mesh. Could you please look into this issue? Thanks in advance.

Steps to Reproduce

python main.py --workspace $model_path --O --test --save_mesh

Expected Behavior

save a textured mesh

Environment

ubuntu 20.04, torch1.13+cu116

tydia commented 1 year ago

Nevermind, I figured out how to fix this bug. The reason that it is not saving mesh is because _export() wasn't called. To fix this, add:

v = torch.from_numpy(vertices).contiguous().float().to(self.aabb_train.device) f = torch.from_numpy(triangles).contiguous().int().to(self.aabb_train.device)

after line 181 in nerf/renderer.py, also add

_export(v, f) at the end of export_mesh() method in nerf/renderer.py

r530044129 commented 1 year ago

@tydia I still meet the error RuntimeError: instance mode - pos must have shape [>0, >0, 4] how do you deal with it?

lukemelas commented 1 year ago

Thanks for finding this! I'll create a PR.

r530044129 commented 1 year ago

@lukemelas thanks! I switch to your PR, but it's still has the error: RuntimeError: instance mode - pos must have shape [>0, >0, 4]