What about distortion parameters calculated by COLMAP? colmap2mvsnet.py assumes undistorted images.

[jlartois]

Someone following the instructions in the "Test on your own data" section of the README will use COLMAP as follows:

// place camera captured images in "images_col" folder
colmap feature_extractor <args>      // extract features
colmap exhaustive_matcher <args>     // match features
mkdir sparse_col
colmap mapper <args>                 // SfM to get the intrinsics, extrinsics and a distortion parameter 'k'
colmap model_converter --input_path sparse_col/0 --output_path sparse_col --output_type TXT

If the camera model is SIMPLE_RADIAL (which is the default), then sparse_col/cameras.txt includes a distortion parameter 'k'. For example 0.062 here: 1 SIMPLE_RADIAL 1920 1080 1707.0 960 540 0.062 The correct next step is then to undistort the images in "images_col":

mkdir "dense"
colmap image_undistorter --image_path images_col --input_path sparse/0 --output_path dense

This results in undistorted images in the dense/images folder, as well as a new dense/sparse/0 folder containing a cameras.bin file, which (when converted to .txt) has different intrinsics and no distortion anymore. For example: 1 PINHOLE 1872 1053 1707.6 1707.6 936 526.5 If someone wanted to use COLMAP for estimating depth maps, they would use the undistorted images, and these new intrinsics. Then, they would like to try this repository as well.

The colmap2mvsnet.py script in this repository however does not take this undistortion step into account. In other words, it assumes that the images in images_col are already undistorted and COLMAP's cameras.bin has distortion parameters all equal to 0.

I think this will result in poor qualitative results when people try out their own dataset using MVSFormer++, which would be unfortunate. Would it be possible to address this in the README.md?

[ewrfcas]

Thanks for your advice! This script did not consider the camera distortion (only containing the sparse reconstruction in colmap without folder dense). Since the new pipeline has a slightly different process, we would update it latter. We also recommend the users to use colmap image_undistorter to undistort custom images if they look unusual.