Code for paper KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereo
Tips: If you meet any problems when reproduce our results, please contact Yikang Ding (dyk20@mails.tsinghua.edu.cn). We are happy to help you solve the problems and share our experience.
Clone this repo:
git clone https://github.com/megvii-research/KD-MVS.git
cd KD-MVS
We recommend using Anaconda to manage python environment:
conda create -n kdmvs python=3.6
conda activate kdmvs
pip install -r requirements.txt
We also recommend using apex
, you can install apex from the official repo.
Download the preprocessed DTU training data (from Original MVSNet), and unzip it to construct a dataset folder like:
dtu_training
├── Cameras
└── Rectified
Download our processed DTU testing data and unzip it as the test data folder, which should contain one cams
folder, one images
folder and one pair.txt
file.
Set the configuration in scripts/run_train_unsup.sh
as:
DATASET_DIR
as the path of DTU training set.LOG_DIR
as the path to save the checkpoints.NGPUS
and BATCH_SIZE
according to your machine.train_unsup.py
, such as summary_freq
, save_freq
, and so on.To train your model, run:
bash scripts/run_train_unsup.sh
Note:
apex
and sync_bn
by default, to use these modules, make sure you have installed the apex
according to the official repo.gipuma
fusion method by default, please make sure you have compiled and installed it correctly. To do so, you need clone the modified version from Yao Yao.
Modify the line-10
in CMakeLists.txt
to suit your GPUs. Then install it by cmake .
and make
, which will generate the executable at FUSIBILE_EXE_PATH
.sync_bn
could help with this problem.To reproduce the results, please note:
validation set
. However, in self-supervised mode, we need to pick models manually based on experience.apex
and sync_bn
could be helpful (but still can't handle this problem completely).Before start training, set the configuration in scripts/run_train_kd.sh
as:
DATASET_DIR
as the path of DTU training set.LOG_DIR
as the path to save the checkpoints.CHECKED_DEPTH_DIR
as the path to our pseudo label folder.NGPUS
and BATCH_SIZE
according to your machine.summary_freq
, save_freq
, and so on.run:
bash scripts/run_train_kd.sh
For easy testing, you can download our pretrained models and put them in ckpt
folder, or use your own models and follow the instruction below.
Make sure:
gipuma
package.Set the configuration in scripts/run_test_dtu.sh
:
TESTPATH
as the path of DTU testing set.CKPT_FILE
as the path of the model weights.OUTDIR
as the path to save results.FUSIBILE_EXE
as the path to gipuma fusible file.Run:
bash scripts/run_test_dtu.sh
The reconstructed point cloud results would be stored in outputs/test_dtu/gipuma_pcd
, you can also download our fused point cloud results of KD-trained model from here.
To get quantitative results of the fused point clouds from the official MATLAB evaluation tools, you can refer to TransMVSNet.
By using the latest code, pretrained model and default parameters, you can get the final results like: | Model | Acc. | Comp. | Overall |
---|---|---|---|---|
unsup | 0.4166 | 0.4335 | 0.4251 | |
KD | 0.3674 | 0.2847 | 0.3260 |
@inproceedings{ding2022kdmvs,
title={KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereo},
author={Ding, Yikang and Zhu, Qingtian and Liu, Xiangyue and Yuan, Wentao and Zhang, Haotian and Zhang, Chi},
booktitle={European Conference on Computer Vision},
year={2022},
organization={Springer}
}
We borrow some code from CasMVSNet and U-MVS. We thank the authors for releasing the source code.