LinHungShi / GCNetwork

117 stars 39 forks source link

Geometry and Context Network (On-Going Project)

A Keras implementation of GC Network by HungShi Lin(hl2997@columbia.edu). The paper can be found here. I do some modifications by adding a linear output function and enable training highway block at the second stage.

Issue

  1. Model performance can't acheive that in the original paper.

Update (10/06/2017)

Model can be trained with image with size (256, 512)

Software Requirement

tensorflow(install from here), keras(install from here)

Data used for training model

I trained my model with drivingfinalpass dataset, which contains more than 4000 stereo images with 2 epochs.

Preprocessing

We crop training patches with size of 256x256 (different from that in the paper) from training images and normalize each channel.

Download

Run the following command:

  git clone https://github.com/LinHungShi/GCNetwork.git

Two ways to download driving dataset:

  1. create subdirectories sceneflow/driving in data, download and tar driving_final pass and driving_disparity from here.
  2. You can also issue command

    “sh download.sh” which will create subdirectories and download datasets.

Train the model by running:

Run the following command:

  python train.py

Predict the data with test.py

  1. create a directory, which contains two subdirectories -- left and right.
  2. Run the following command

    python test.py -data <path/to/directory> -wpath <path/to/weight> [option]

  3. The default file will be saved as npy file named prediction.npy, you can replace it with -pspath when issuing the above command.

    (Optional) Specify the pretrained weight by

  4. Set it in train_params.py
  5. python train.py -wpath

Something you might want to do

  1. To enable training with Monkaa dataset, a. Download Monkaa dataset from previous link.

    b. Create a directory in data, which has the name as monkaa_root in src/environment.json.

    c. Create a subdirectory, which has the name as monkaa_train in src/environment.json.

    d. Create a subdirectory, which has the name as monkaa_label in src/environment.json.

  2. All hyperparameters used for building the model can be found in src/hyperparams.json

Reference :

Kendall, Alex, et al. "End-to-End Learning of Geometry and Context for Deep Stereo Regression." arXiv preprint arXiv:1703.04309 (2017).