question about split of seen and unseen

[ruinianxu]

Hi,

First of all, thanks for sharing the amazing dataset.

I mainly have a question about the way of splitting the seen and unseen sets. I am sorry if I miss something already presented in the paper. Since the dataset is about the entire indoor environment with a lot of objects, I wonder how to define if this task is seen. Is it about the object, environment or both?

Really appricate your answer.

[SouLeo]

Hello,

I think they briefly touch on this in the paper here

" we further split the validation and test into two conditions: seen and unseen environments. This split facilitates examining how well models generalize to entirely new spaces with novel object class variations."

These were the last couple of sentences in section 3.1. So I'm inferring that both the environment and objects are different in the unseen scenarios.

Best of luck!

[ruinianxu]

@SouLeo Hi，

Really appreciate your answer. I am still curious if novel object represents the novel instance or category. For example, one type of apple is in the training while another type of apple is in the unseen test set. Or apple in the training while banana in the unseen testing.

[MohitShridhar]

'Unseen' primarily refers to new rooms which the agent has never seen. There are no novel object categories in the unseen split, but there are unseen instances, e.g red apple in training scenes, green apple in unseen test scenes, or a book with a completely different texture/title.

[ruinianxu]

@MohitShridhar Got it. Thank you so much for your answer.