pytorch-superpoint

This is a PyTorch implementation of "SuperPoint: Self-Supervised Interest Point Detection and Description." Daniel DeTone, Tomasz Malisiewicz, Andrew Rabinovich. ArXiv 2018. This code is partially based on the tensorflow implementation https://github.com/rpautrat/SuperPoint.

Please be generous to star this repo if it helps your research. This repo is a bi-product of our paper deepFEPE(IROS 2020).

Differences between our implementation and original paper

• Descriptor loss: We tested descriptor loss using different methods, including dense method (as paper but slightly different) and sparse method. We notice sparse loss can converge more efficiently with similar performance. The default setting here is sparse method.

Results on HPatches

• Pretrained model is from SuperPointPretrainedNetwork.
• The evaluation is done under our evaluation scripts.
• COCO/ KITTI pretrained model is included in this repo.

Installation

Requirements

• python == 3.6
• pytorch >= 1.1 (tested in 1.3.1)
• torchvision >= 0.3.0 (tested in 0.4.2)
• cuda (tested in cuda10)

conda create --name py36-sp python=3.6
conda activate py36-sp
pip install -r requirements.txt
pip install -r requirements_torch.txt # install pytorch

Path setting

• paths for datasets ($DATA_DIR), logs are set in setting.py

Dataset

Datasets should be downloaded into $DATA_DIR. The Synthetic Shapes dataset will also be generated there. The folder structure should look like:

datasets/ ($DATA_DIR)
|-- COCO
|   |-- train2014
|   |   |-- file1.jpg
|   |   `-- ...
|   `-- val2014
|       |-- file1.jpg
|       `-- ...
`-- HPatches
|   |-- i_ajuntament
|   `-- ...
`-- synthetic_shapes  # will be automatically created
`-- KITTI (accumulated folders from raw data)
|   |-- 2011_09_26_drive_0020_sync
|   |   |-- image_00/
|   |   `-- ...
|   |-- ...
|   `-- 2011_09_28_drive_0001_sync
|   |   |-- image_00/
|   |   `-- ...
|   |-- ...
|   `-- 2011_09_29_drive_0004_sync
|   |   |-- image_00/
|   |   `-- ...
|   |-- ...
|   `-- 2011_09_30_drive_0016_sync
|   |   |-- image_00/
|   |   `-- ...
|   |-- ...
|   `-- 2011_10_03_drive_0027_sync
|   |   |-- image_00/
|   |   `-- ...

• MS-COCO 2014
  • MS-COCO 2014 link
• HPatches
  • HPatches link
• KITTI Odometry
  • KITTI website
  • download link

run the code

• Notes:
  • Start from any steps (1-4) by downloading some intermediate results
  • Training usually takes 8-10 hours on one 'NVIDIA 2080Ti'.
  • Currently Support training on 'COCO' dataset (original paper), 'KITTI' dataset.
• Tensorboard:
  • log files is saved under 'runs/<\export_task>/...'

tensorboard --logdir=./runs/ [--host | static_ip_address] [--port | 6008]

1) Training MagicPoint on Synthetic Shapes

python train4.py train_base configs/magicpoint_shapes_pair.yaml magicpoint_synth --eval

you don't need to download synthetic data. You will generate it when first running it. Synthetic data is exported in ./datasets. You can change the setting in settings.py.

2) Exporting detections on MS-COCO / kitti

This is the step of homography adaptation(HA) to export pseudo ground truth for joint training.

• make sure the pretrained model in config file is correct
• make sure COCO dataset is in '$DATA_DIR' (defined in setting.py)
• config file:

  export_folder: <'train' | 'val'>  # set export for training or validation

  General command:

  python export.py <export task>  <config file>  <export folder> [--outputImg | output images for visualization (space inefficient)]

  export coco - do on training set

  python export.py export_detector_homoAdapt configs/magicpoint_coco_export.yaml magicpoint_synth_homoAdapt_coco

  export coco - do on validation set

• Edit 'export_folder' to 'val' in 'magicpoint_coco_export.yaml'

  python export.py export_detector_homoAdapt configs/magicpoint_coco_export.yaml magicpoint_synth_homoAdapt_coco

  export kitti

• config
  • check the 'root' in config file
  • train/ val split_files are included in datasets/kitti_split/.

    python export.py export_detector_homoAdapt configs/magicpoint_kitti_export.yaml magicpoint_base_homoAdapt_kitti