ashawkey / nerf2mesh

[ICCV2023] Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement
https://me.kiui.moe/nerf2mesh/
MIT License
879 stars 87 forks source link

Training failed for high number of iterations in stage 0 with default settings in sdf mode #105

Open PaulSK98 opened 3 months ago

PaulSK98 commented 3 months ago

Hello, thanks for your amazing work. I'm using your repo and the paper for textured mesh generation of custom objects for an object detection pipeline. I build lots of meshes and got some amazing results with the default settings:

Results for default setting

Here is an example of casual generated from casual smartphone images: nerf stage 1: https://github.com/ashawkey/nerf2mesh/assets/108192596/585dba7d-cc92-4689-9b84-2bc3d3269518 mesh: Screenshot from 2024-03-08 15-03-31

Obeservations an Problem

I did some hyperparameter experiments for different objects to see how the results are influenced. I had a similar problem for the most objects when changing the iterations in stage zero to a higher value (15000 and upwards): At some epoch the training fails and the training error is NaN. Here is a screenshot and the corresponding log file for stage zero with 30000 iterations (the rest is default setting for single object reconstruction) Screenshot from 2024-03-08 15-32-04

Here is a log file from stage 0 for the elephant i showed above, when increasing the iterations to 30000: log_ngp_stage0.txt I used the recommended default setting for both stages for single object reconstruction in --sdf mode.

I saw the same behaviour for different objects at some point where i increased the number of iterations. For some objects (esp. with some shiny surfaces) I even saw this for the default setting. All datasets were created using COLMAP, The sparse point cloud and the camera location are reasonable.

Did someone noticed a similar behavior or can explain this? I am grateful for any input or tips and can provide additional input and results

keep up the grate work Paul