search

TRI-ML / packnet-sfm

TRI-ML Monocular Depth Estimation Repository
https://tri-ml.github.io/packnet-sfm/
MIT License
1.24k stars 243 forks source link

Batch Size > 1 with NRS Network #121

Closed jdriscoll319 closed 3 years ago

jdriscoll319 commented 3 years ago

In the NRS paper you call out using a batch size of 4 across all training experiments (KITTI, dashcam, etc). However when I try to run training with a batch size > 1 the systems crashes and exits. I worked out that this is happening in camera_generic.py and have fixed it. While the system no longer crashes it doesn't seem to be learning correctly. Loss is dropping but the depthmaps look like crap compared to results from batch size 1 training.

Given all of this I'm wondering where the discrepancy between the shipped code being unable to handle batch size > 1 and the paper referencing batch size 4 is coming from. Is this a typo in the paper? Is there something inherent to the way the network is constructed that prevents it from learning when batch size is greater than 1? Or is there another version of the code that hasn't been released?

jdriscoll319 commented 3 years ago

Looks like I got it working. There were some additional issues with the view synthesis process that I needed to correct

VitorGuizilini-TRI commented 3 years ago

Thank you for the heads up, I will take a look at NRS with batch size > 1. If you don't mind you can start a PR with the fixes, I can take a look as soon as possible.

jdriscoll319 commented 3 years ago

https://github.com/TRI-ML/packnet-sfm/pull/123