Learning Motion Flows for Semi-supervised Instrument Segmentation from Surgical Robotic Videos

by Zixu Zhao,Yueming Jin, Xiaojie Gao, Qi Dou, Pheng-Ann Heng.

Introduction

• The Pytorch implementation for our MICCAI 2020 paper 'Learning Motion Flows for Semi-supervised Instrument Segmentation from Surgical Robotic Videos'.

• The Code contains two parts: motion learning (flow prediction and flow compensation) and semi-supervised segmentation.

Data Preparation

• We use the dataset from MICCAI 2017 Endoscopic vision challenge.

• Please follow the data preprocessing steps in this repository.

• The structure of data folder is arranged as follows:

  
  (root folder)
  ├── data
  |  ├── cropped_train
  |  |  ├── instrument_dataset_1
  |  |  |  ├── images
  |  |  |  ├── binary_masks
  |  |  |  ├── parts_masks
  |  |  |  ├── instrument_masks
  |  |  ├── instrument_dataset_2
  |  |  |  ├── images
  |  |  |  ├── binary_masks
  |  |  |  ├── parts_masks
  |  |  |  ├── instrument_masks
  |  |  ├── ......

### Setup & Usage for the Code

1. Check dependencies:

• Python 3.6
• pytorch 0.4.1+
• pytorch-ignite 0.2.0+
• tensorboardX
• albumentations
• opencv-python
• cupy (please check your CUDA version before install)
• tqdm

  
  2. Flow prediction & Flow compensation (./motion_learning/):

• To train the flow prediction model, run $ python train_mp.py.
• To train the flow compensation model, run $ python train_mc.py.
• Arguments for model training in train.sh are in default settings.
• To propagate frame-label pairs, run $ bash propagate.sh.

3. Semi-supervised segmentation (./segmentation/):

   $ bash train.sh

   Note: You may try other models from /Models/plane_model.py

Citation

If this repository is useful for your research, please cite:

@inproceedings{zhao2020learning,
  title={Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video},
  author={Zhao, Zixu and Jin, Yueming and Gao, Xiaojie and Dou, Qi and Heng, Pheng-Ann},
  booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
  pages={679--689},
  year={2020},
  organization={Springer}
}

Questions

For further question about the code or paper, please contact 'zxzhao@cse.cuhk.edu.hk'