Neus2

[cdcseacave]

Currently Neus is implemented, but its successor, Neus2, is promising improved accuracy and faster processing. I am quite well equipped on the photogrammetry side of things to have an attempt myself at extending nerfstudio to support this method (I am the developer of OpenMVS and I watched closely the progress of NeRFs too), but I have little practical experience with ML, and nerfstudio. Is there someone that could guide me in implementing this?

[niujinshuchong]

I think you could refer to the current implementation of NeuS in the repo and add the coarse to fine schedule of hash encoding. But you need to update the tiny-cuda-nn code base if you want to implement their second order gradient computation of hash encoding. We also have our own implementation of second order gradient of hash encoding in MonoSDF https://github.com/autonomousvision/monosdf/tree/main/code/hashencoder. Dynamic scene is not that straightforward to implement into the current codebase. Maybe you could refer to nerfstudio which seems supports dynamic nerf already.

[juuso-oskari]

I think you could refer to the current implementation of NeuS in the repo and add the coarse to fine schedule of hash encoding. But you need to update the tiny-cuda-nn code base if you want to implement their second order gradient computation of hash encoding. We also have our own implementation of second order gradient of hash encoding in MonoSDF https://github.com/autonomousvision/monosdf/tree/main/code/hashencoder. Dynamic scene is not that straightforward to implement into the current codebase. Maybe you could refer to nerfstudio which seems supports dynamic nerf already.

Actually the hash encoding part has the double-backward already implemented as described in https://github.com/NVlabs/tiny-cuda-nn/issues/58. But the problem is the fully fused MLP after that, which is supposed to produce the sdf. Does this network produce it as well or does it only do the hash encoding?