search

zhyever / PatchRefiner

[ECCV 2024] Leveraging Synthetic Data for Real-Domain High-Resolution Monocular Metric Depth Estimation
MIT License
34 stars 5 forks source link

Data list for quantitative comparison #5

Closed Byeongjun1022 closed 3 weeks ago

Byeongjun1022 commented 3 weeks ago

Hello, thank you for sharing your great research and releasing the code.

I would like to ask whether the test set used for the quantitative comparison on the UnrealStereo4K dataset is 'splits/test_in.txt' or 'splits/test_out.txt'. Additionally, could you please clarify which test set was used for the quantitative comparison on the CityScapes dataset?

Thank you!

zhyever commented 3 weeks ago

Hi, thanks for your interest in our work.

We use val.txt to evaluate the model as default following previous metric depth papers. The numbers and in-domain results in our paper are all from this set. The text_in.txt indicates in-domain test set, which could also be good for evaluation but we just left it here. The text_out.txt is a set for out-of-domain evaluation. The model performs much worse here, indicating some generalization issues waiting for solutions.

For the CityScapes, the training set includes the standard training data plus the extra set. The validation set is also just the official one. I put the corresponding files right below. Note that the evaluation protocal presented in this paper is slightly different from the original paper. As we claimed, the current one makes much more sense and we encourage researchers to report the updated results in their paper for fair comparisons. Please check here for more information.

train.txt val.txt

Btw, I just noticed some small issues in the cityscapes training docs. I'm working on it and I will also add the specific number for comparison for the cistyscapes. I will leave a message here after finishing it.

Byeongjun1022 commented 3 weeks ago

Thanks for fast reply. It is really helpful.