Are the parameters in eval_tanks.sh inconsistent with that on papers? Thanks

[mazeyu]

Are the parameters in eval_tanks.sh inconsistent with that on papers? Thanks I tried increasing the max h max w, or reducing the threshold. But still only around 0.7 f2score for the reference Ignatius scene, which can be around 0.8 according to the official Tanks website.

[weizizhuang]

This version of the code has been jointly developed by our team, some parameter settings may be different from the paper due to insufficient communication. In my test, only half size of the image resolution was used for TNT evaluation with D=512 inverse setting. The mean F-score could be over 60 with training on BlendedMVS, which is the best one among all the open source methods.

[mazeyu]

Thanks for your reply. So, this means the parameters in the code are all good, and I should be able to get the results on Tanks benchmark, right? Regarding the weights, you mean we can get 59 for DTU pretrained and over 60 for Blended pretrained, right? Thanks

[weizizhuang]

You can find the submissions on Tanks and Temples leaderboard with different pretrained models, “test_d2hcrmvsnet_release” got 58.62 while "test_d2hcrmvsnet_blended" got 60.13 by running this code directly. There may be some thing wrong when you test the reference Ignatius scene yourself, I guess it is the wrong .log file, which is used to align point clouds.

[mazeyu]

Wow. In my test, I use the camera pose computed using COLMAP by myself. If you have those poses for Tanks training set, could you share with me, thank you very much! If not, I will try to figure out if my poses have problems.

P.S. I noticed the official website has provided raw images and poses, let me know if you preprocessed them and use new poses, or not? Thanks ! :)

[QT-Zhu]

We actually borrow the preprocessed TnT training set of original MVSNet repo. The google drive link is here.

[mazeyu]

Great! It is very helpful! Thank you so much