Test doesn't work on Tanks and Temples

[murrdpirate]

I followed the instructions on the README and get good outputs for the DTU test set, but when I test on TNT, I just get distorted depth maps, as shown below.

I downloaded the linked TNT dataset and the two trained checkpoints (DTU and finetune on BLD). Both checkpoints fail for me on TNT.

[TQTQliu]

Hello, thanks for your interest in our work. Did you use the short depth range version of the camera parameters for evaluating the Intermediate set? Otherwise, you may get a distorted depth map. Please refer to here

[murrdpirate]

Hi, thanks for getting back to me so quickly. Yes, I did use the short depth range version. I also had to rename the cams directories to cams_1.

[TQTQliu]

I suspect it may be a problem with the visualization code you're using. You can run bash scripts/test_tnt_inter.sh exp_name --save_jpg to get visualization of the depth maps, but first you need to comment out lines 444 to 451 in test_dypcd_tnt_inter.py, because we did not use the monocular depth branch in MVSTER. You can get the following reasonable results. We will further improve the code and readme in the future, best wishes！

[murrdpirate]

I tried the visualization you mentioned, but just get images like this, below. The point clouds are also incorrect.

I'm sure there's just something being done wrong, but I can't tell if it's on my end. Appreciate your efforts.

[TQTQliu]

I am sorry to hear that. I cloned the repository again and downloaded the pre-training weights, I can get reasonable depth map visualization.

[TQTQliu]

You could try other MVS methods and see if they can produce a reasonable depth map, which may help to locate the problem.

[ruyanyinian]

@TQTQliu Great work on MVS. I got the same problem as @murrdpirate met, when I tested tnt Panther. I used the model finetuned on blendMVS, and camera parameter was short range. I got no idea about this

[ruyanyinian]

@murrdpirate Did you solve this?

[TQTQliu]

@ruyanyinian Hi, I can get reasonable results on my server, such as the Panther scene you mentioned. I'm not sure if it's the environment that's causing it, you could try checking. The python version I'm using is 3.10.8.

certifi==2023.11.17
charset-normalizer==3.3.2
idna==3.6
joblib==1.3.2
numpy==1.26.2
opencv-python==4.8.1.78
packaging==23.2
Pillow==10.1.0
plyfile==1.0.2
protobuf==4.25.1
requests==2.31.0
scikit-learn==1.3.2
scipy==1.11.4
tensorboardX==2.6.2.2
threadpoolctl==3.2.0
torch==1.13.1+cu116
torchaudio==0.13.1+rocm5.2
torchvision==0.14.1+cu116
typing_extensions==4.9.0
urllib3==2.1.0

[ruyanyinian]

@TQTQliu Thanks for quick reply. Maybe environment difference maybe the cause for getting corrupt depth map, I will keep align with your env requirement and try it again. By the way, just for clear, If I test on tanksandtemple Intermediate level, I need to use short range of cam parameter, and change its name with "cam_1", for every corresponding scene, right?

[TQTQliu]

Yes! The structure is just like：

tanksandtemples                          
       ├── intermediate
              ├── Family
                       ├──images
                       ├──cams_1
                        pair.txt
                        Family.log
              ├── ...  

[ruyanyinian]

@TQTQliu I had my env same with you and re-clone your work and model. However, I didn't get correct depth map when testing tnt both for intermediate and advance. For DTU testing, it was good and no problem with it.

[TQTQliu]

It's really strange. I'll compress the TT data set on my server, upload it to Google Cloud disk, and then give you the download link, maybe you can try it.

[ruyanyinian]

@TQTQliu Awesome!!! Thanks a lot, and I'm really appreciated if did so

[TQTQliu]

@ruyanyinian Hi, the download link for the TT dataset is provided here.

[murrdpirate]

Thank you @TQTQliu ! And @ruyanyinian for helping figure out the issue.

The updated data seems to work for me. It looks like there was an issue in the last line of the cam.txt files. I think the original version had min val and step size instead of min and max values.

[ruyanyinian]

@murrdpirate @TQTQliu I tried modified version of tnt dataset, and got correct depth map as well. As @murrdpirate mentioned, maybe there is gap between modified data and original one.

[TQTQliu]

@murrdpirate @ruyanyinian Thanks for your feedback, I have added this download link in the readme.

[05063112lcs]

感谢您的快速回复。也许环境差异可能是导致深度图损坏的原因，我会与您的环境要求保持一致，然后再试一次。顺便说一句，为了清楚起见，如果我在tanksandtemple中级水平上测试，我需要使用短距离的cam参数，并将其名称更改为“cam_1”，对于每个相应的场景，对吧？ Hello, I saw the question you posted in the ET-MVSNet project, I have a question that I need to ask, can you reproduce the quantitative results of the TNT dataset in the paper, why the indicators I got are very poor, very poor.