search

facebookresearch / ImageBind

ImageBind One Embedding Space to Bind Them All
Other
8.38k stars 771 forks source link

Inconsistent Statement Regarding Experiments on NYU-Depth-v2 #88

Closed StanLei52 closed 1 year ago

StanLei52 commented 1 year ago

There seems to be an inconsistency in the documentation regarding experiments conducted on the NYU-Depth-v2 dataset, specifically related to the dataset split used for evaluation.

In Table 1 of your paper, it is stated that the size of the testing set for NYU-Depth-v2 is 653. However, I noticed an inconsistency in the dataset split details. In Appendix A, it is mentioned that the evaluation was conducted on the 794 validation set. As per my understanding, NYU-Depth v2 training set consists of 794 samples, including the other class, which corresponds to the 18 different semantic classes. In contrast, the validation set contains only 7 other classes, with 653 samples in total.

To ensure clarity and accuracy, I would appreciate it if you could clarify which dataset split your reported results are based on.

StanLei52 commented 1 year ago

@imisra @rohitgirdhar

imisra commented 1 year ago

Hi @StanLei52

Thanks for your comment! It is a typo in the Appendix A and Table 1 is correct. We use the 653 test set samples for evaluation.