m-niemeyer
commented
4 years ago Hi @kwea123 , thanks a lot for your interest in the project!
Yes, what you observed is indeed very interesting. The main focus of our work is to learn to reconstruct textured objects while having only posed multi-view images with object masks and possibly depth maps as supervision. We choose to represent the object implicitly, i.e. f(x) = tau. In contrast, in NeRF the main focus is multi-view consistent novel-view synthesis, also only from posed multi-view images. I think both representations are very interesting and have pros and cons. In addition, this is a very active research area - I am also very excited to see where this line of research goes.
Regarding different threshold values, we used the obvious choice of 0.5 in this project. In our previous project Occupancy Networks, we learned implicit functions with 3D supervision and we show and ablation study in the supplementary material (e.g. Fig. 3 - 4). Here, the value 0.2 performed best. However, the loss which is optimized is different so it cannot directly be compared.
I hope this helps a little. Good luck with your research!
I recently read NeRF, which is a concurrent work with the similar idea. In their work they use the "expected distance", which is a weighted average of all sampled points on the ray, as the final distance; currently they don't use depth to train, it's just for visualization. Then I came across your paper whose depth estimation is more "correct" in my intuition, which is to define the depth as the distance at which the ray stops.
So here is my question, did you compare different tau values? Which one performs the best in terms of depth prediction? And what's your opinion on the "expected depth" (NeRF) and the "thresholded depth" (your work)?