search

tinghuiz / SfMLearner

An unsupervised learning framework for depth and ego-motion estimation from monocular videos
MIT License
1.97k stars 557 forks source link

Single view depth has limitation of generalizing #50

Closed evakasch closed 6 years ago

evakasch commented 6 years ago

I think that single view depth has more problems generalizing to previously unseen types of images. For example, when I rotate the input image 180 degrees, the depth output is totally wrong. The input of two images is necessary to get robust results.

tinghuiz commented 6 years ago

I agree. Single-view and stereo depth estimation are two very different problems. For single-view, the model can only rely on semantic cues specific to the training set. Therefore, it's unreasonable to expect the model to generalize to test scenes that have dramatically different statistics than the training scenes (e.g. a model trained on outdoor driving sequences is unlikely to work well on indoor scenes or the test images are rotated 180 degrees while training images are not). On the other hand, the stereo model can utilize not only the semantic cues but also geometric cues for depth estimation, which are much more generalizable.