Performance of DReg-NeRG on low- or zero overlapping objects

[ttsesm]

Hi,

Congrats for your work and thanks for making it available.

As the title of the issues points out, I would like to ask whether you have done any experiments or tested how your solution performs on low- or zero overlapping objects.

For example imagine that you have two individual pieces of a complete object (broken object in two pieces), and for each you create a NeRF model as you do in your case scenario and you would like to apply an assembly task (i.e. registration with low- or no-overlap). Would your approach work in such a case scenario or would be possible to be modified for such a case?

Thanks.

[AIBluefisher]

Hi, thanks for your interest in our work. We actually did not test on low/zero -overlapping objects. It is difficult to register NeRF under low-zero overlapping scenes since it poses challenges to the generalizability of NeRF. NeRF is good at interpolation while performing badly on extrapolation. As our method relies on the occupancy grid that is trained by posed images, the performance can be bad if the occupancy grid is under-determined in unseen space. It is also possible to modify our method to let it work on the low/zero -overlapping settings by regularizing the NeRF by sparse/dense depth -- this improves the generalizability of NeRF on the few-shot setting. But I don't think it can work very well.