search

autonomousvision / unimatch

[TPAMI'23] Unifying Flow, Stereo and Depth Estimation
https://haofeixu.github.io/unimatch/
MIT License
1.05k stars 106 forks source link

Depth estimation from images without any pose information #8

Closed spiegelball closed 1 year ago

spiegelball commented 1 year ago

Very nice results! Is it possible to generate depth information from an image sequence which has no pose information at all?

haofeixu commented 1 year ago

Unfortunately not for our method, we need the pose information to model the geometry constraints for feature matching.

renwuli commented 1 year ago

Hi @haofeixu, with respect to the demo of depth estimation, using the same settings as the colab notebook, after adding some noise to the rotation part of pose while keeping the translation part the same, I found the depth quality drops a lot. Similarly, after adding some noise to the translation part of pose while keeping the rotation part the same, the depth quality drops a lot as well. Seems like the depth estimation head is not robust to pose. Any ideas to handle this?

FreakTheMighty commented 5 months ago

@spiegelball @haofeixu are poses used when estimating flow? It didn't seem to me that they were. If not, then it would seem to me this could be done in two stages