search

yzcjtr / GeoNet

Code for GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose (CVPR 2018)
MIT License
726 stars 181 forks source link

Joint training depth and flow #24

Closed SeokjuLee closed 6 years ago

SeokjuLee commented 6 years ago

Hi, thanks for sharing codes! I have a question for joint learning depth and flow networks. Are there some positive effects when training the tasks all together?

I wonder why you didn't train them all at the last phase.

yzcjtr commented 6 years ago

Hi @SeokjuLee , actually I experimented with finetuning all the networks together but found no improvement. This is explained in the Training Details in the experimental part of our paper:

Though the sub-networks can be trained together in an end-to-end fashion, there is no guarantee that the local gradient optimization could get the network to that optimal point. Therefore, we adopt a stage-wise training strategy

SeokjuLee commented 6 years ago

@yzcjtr Thanks for the reply. So the ResFlowNet doesn't affect the performance of depth estimation. Is this right? Table 1 shows GeoNet performs significantly better than SfM-Learner [56]. Do SSIM loss and network structure (vgg, resnet) make this huge performance gap?

yzcjtr commented 6 years ago

Yes. Both loss and architecture account for the improvement. You can see detailed analysis in Sec 4.2 of our paper.

SeokjuLee commented 6 years ago

@yzcjtr Thank you!