Notice there are three scenes missing in the test set?

[ch3cook-fdu]

Missing 'scene0789_00', 'scene0792_00', 'scene0806_00' for testing dense captioning, is predictions on these scenes required?

[daveredrum]

Hi @ch3cook-fdu, those scenes after ``scene0707_00'' are from the hidden test set. You don't need those scenes for training and validation, unless you would like to submit your predictions to the public benchmark.

[ch3cook-fdu]

yes i'm testing the network for submission, and i notice that the generated file consists of 97 scenes rather than 100, which is the size of whole scannet test set.

is it normal?

[daveredrum]

Those scenes should be included in the submission. Could you check if you downloaded the whole ScanNet v2?

[ch3cook-fdu]

I notice that it's because the scanrefer test set only contains annotations for 97 scenes in the test set, so would i upload results for 97 scenes or 100 scenes?

[daveredrum]

Oh, then it's definitely okay to omit those three scenes - Sorry for the confusion. I totally forgot that we excluded those scenes, probably because there is limited context to work on in those scenes.

[ch3cook-fdu]

understood! thanks for clarification!