In this paper, we address the problem of estimating dense depth from a sequence of images using deep neural networks. Specifically, we employ a dense-optical-flow network to compute correspondences and then triangulate the point cloud to obtain an initial depth map. Parts of the point cloud, however, may be less accurate than others due to lack of common observations or small baseline-to-depth ratio. To further increase the triangulation accuracy, we introduce a depth-refinement network (DRN) that optimizes the initial depth map based on the image’s contextual cues. In particular, the DRN contains an iterative refinement module (IRM) that improves the depth accuracy over iterations by refining the deep features. Lastly, the DRN also predicts the uncertainty in the refined depths, which is desirable in applications such as measurement selection for scene reconstruction. We show experimentally that our algorithm outperforms state-of-the-art approaches in terms of depth accuracy, and verify that our predicted uncertainty is highly correlated to the actual depth error.
For convenience, the code are assumed to be run inside NVIDIA-Docker. For instructions on installing NVIDIA-Docker, please follow the following steps (note that this is for Ubuntu 18.04):
For more detailed instructions, please refer to this link.
Install Docker
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent \
software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
To verify Docker installation, run:
sudo docker run hello-world
Install NVIDIA-Docker
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
sudo apt-key add -
sudo apt-get install nvidia-docker2
sudo pkill -SIGHUP dockerd
To activate the docker environment, run the following command:
nvidia-docker run -it --rm --ipc=host -v /:/home nvcr.io/nvidia/pytorch:20.03-py3
where /
is the directory in the local machine (in this case, the root folder), and /home
is the reflection of that directory in the docker.
This has also specified NVIDIA-Docker with PyTorch version 20.03 which is required to ensure the compatibility
between the packages used in the code (at the time of submission).
Inside the docker, change the working directory to this repository:
cd /home/PATH/TO/THIS/REPO/DeepMultiviewDepth
Please download and extract the files provided by this link to ./checkpoints
directory.
Run demo.sh
to get the results of the demo dataset in ./viusalization
folder.
The results contain inputs, intermediate outputs, and final outputs of the proposed pipeline, same as the figure below: