Training NeuS with large scale scenes

[wangyida]

Thanks for your nice work, my reconstruction targets are real outdoor scenes represented in scale of about 10 to 50 meters for each dimension. Nerf could be trained properly with hash encoding, but NeuS won't be trained correctly, which leads to train/inv_s=5 when model is converged. I have changed model.radius to 10 so that the ray could go through the correct space.

Should I change other hyper-parameters accordingly to get the NeuS model fit for large scenes? Should I increase sphere_init_radius accordingly as well?

[bennyguo]

Hi! What kinds of outdoor scenes are you training on? Are they unbounded 360 scenes (cameras looking at an object, with background) or unbounded scenes (like street views)?

[wangyida]

It's unbounded scene like street views, there are no masks included

[bennyguo]

Adopting NeuS on street views can be quite challenging as the buildings are somehow seen from sparse views and the scene is unbounded (unlike indoor scenes). There are some thoughts that might be useful:

• Change the SDF initialization to be a large sphere approximately tangent to the ground;
• Apply a NeRF-based background model (I'm also working on this);
• Utilize monocular depth/normal cues like in NeuRIS.

[imbinwang]

Hi @wangyida , I'm now training Neuralangelo on similar outdoor sequences, the result mesh has many floaters and holes. The geometric init of VolumeSDF seems critical to the convergence and qulity, how did you set the sphere_init_radius and init_val?