Hello,
I recently took a closer look at the objects used in your dataset.
I found that the scenes 188 and 189 are completely comprised of objects which were already used in other splits (scene 188 from test_seen and scene 189 from test_similar).
I assume there was a mixup when selecting objects for the scenes, but for using the test_novel split to evaluate the generalization quality of models this should be addressed.
Hello, I recently took a closer look at the objects used in your dataset. I found that the scenes 188 and 189 are completely comprised of objects which were already used in other splits (scene 188 from test_seen and scene 189 from test_similar). I assume there was a mixup when selecting objects for the scenes, but for using the test_novel split to evaluate the generalization quality of models this should be addressed.