maturk
commented
2 months ago Hi @pwais and thanks for your interest :) So I tested the following methods for monodepth-to-sfm alignment:
1.) solve for a per frame shift and scale using gradient descent as described in the paper DepthRegGS 2.) solve for a per frame shift and scale using the closed form solution. This closed form solution is the same that is used in earlier MonoSDF work and also originates from the MiDAS paper.
Regarding the performance of the two methods, I found that the results from the gradient descent method, although in general worked well and I am able to reproduce results similar to the DepthRegGS paper, could result in some annoying outliers when compared to the direct closed-form solutions. So in the end, the closed-form method was chosen because naturally there is no randomness in those solutions.
I also have some legacy code in the scripts to support data from sdfstudio (which do not come with colmap database files, but instead has per frame SfM point files) which could be extended later to support any input dataset, not just COLMAP processed ones.
Hope this helps a bit.
pwais
It looks like in your https://github.com/maturk/dn-splatter/blob/main/dn_splatter/scripts/align_depth.py script you might have experimented with a few different monodepth-to-sfm alignment methods... and perhaps settled on a method / code derived from MonoSDF (or MiDaS?) ( https://github.com/autonomousvision/sdfstudio/pull/76/files#diff-dbb421d9d67076bcdfbcd7136b436e0eb4b6801fee775609c2298bd6788dff69R44 https://github.com/nerfstudio-project/nerfstudio/blob/d9b022b25f53f879a34dd522c5ea5b0fe9cf04f7/nerfstudio/utils/math.py#L325 ... )
Your paper briefly mentions it in Section 4.1, but was wondering if you might comment on e.g. any experimental validation that you did?
cc @maturk mebbe
Really cool paper! ☃️