Ra-NeuS

[wangyida]

Adding a new up_est_method z-axis, so that the normalized poses could be always with an averaged up-direction of [0,0,1] as mentioned by nerf2mesh, even when ground is missing in the training data. Such z-axis behaves similarly with ground when ground is included in the training data.

z-axis up-direction

• [x] z-axis test for - ✅ mipnerf360 ✅ synthetic nerf
• [x] mix & match z-axis with - ✅ camera ✅ point ✅ lookat
• [x] test for pose match with nerf2mesh

Extensions

• [x] SDF-to-radiance projection proposed by HF-NeuS and NSR
• [x] weight/alpha regularization
• [x] add a configurable scale factor for camera translation vectors
• [x] directly load resized images (2, 4, *8) when they are available
• [x] appearance embedding

[bennyguo]

Hi! It seems that this new up_est_method rotates the average camera up direction to +z. May I ask how this is different from up_est_method == 'camera'?

[wangyida]

Hi Benny, the difference is that when camera is not positioned evenly surrouding the central object, e.g. the distance is not the same, camera setup will make the object tilted against z axis. the new axis-z setup will still position the object straight up along z axis.

[bennyguo]

I see. Could you keep the config files unchanged and only modify the dataset? Also, is there an example scene illustrating the difference between the two methods so that I can test (for example some scene in Mip-NeRF360 dataset)?

[wangyida]

Sure I can do it on bonsai scene

[wangyida]

When ground is included, the new up direction option axis-z behaves similarly with ground. BTW you may need a light github theme to view this figure, the captions are with transparent background.

[bennyguo]

Sorry I totally forgot about this PR 😂 Thank you very much for the newly added features, and I'll look into the code soon.

[wangyida]

Guess a naive implementation for point center should be conditioned on points of sparse MVS reconstruction instead of cameras @bennyguo

[wangyida]

@bennyguo there are some new functions added, you mind review a bit further to see which of these modules are useful?