monodepth360

Master's project implementing depth estimation for spherical images using unsupervised learning with CNNs. Based on the work of Godard et. al.

Requirements

Requires typical scientific computing libraries like numpy, scipy and TensorFlow. Some Python scripts require other libraries like Blender or OpenEXR. The main program for training and testing just needs TensorFlow. Requires a GPU for training in a reasonable amount of time (inference is okay on CPU).

Structure

• calibration
  • MATLAB scripts for rectifying a top and bottom spherical image.
• evaluation
  • Code for evaluation predictions against ground truth data. Requires PILLOW, matplotlib, numpy, scipy and OpenEXR.
• monodepth
  • Main code section which includes a spherical image module with TensorFlow functions for converting between different projection models. Also includes network definitions and training and testing code. Further instructions are given in the module README
• synthetic
  • Blender scripts for rendering equirectangular images along with ground truth depth (and optionally 3D 360 degree images). Also includes SUNCGToolbox for handling data from their paper. Allows you to create a training dataset for the network. Real world data is currently not made publicly available.
• visualization
  • Code for handling outputs from the network i.e. converting from depth map to point cloud, and resampling the depth map.