Possible inconsistency in objectnav dataset

[srama2512]

❓ Questions and Help

I noticed that objectnav agents do not get rewarded for visiting certain objects from the target category. I wanted to investigate this further, and see if there's a difference b/w the set of objects present in the objectnav task dataset, and the set of objects in the scene.

Here are a set of object categories from scene 1pXnuDYAj8r in MP3D. All these objects have a set of instances that are missing from the objectnav dataset.

Object name: cabinet | Missing ids: {875, 430, 432, 434, 595}
Object name: bathtub | Missing ids: {207}
Object name: towel | Missing ids: {269, 942, 175, 213, 184, 30}
Object name: picture | Missing ids: {512, 897, 142, 143, 785, 146, 275, 786, 787, 788, 155, 159, 671, 289, 290, 291, 804, 550, 556, 301, 820, 181, 826, 699, 315, 955, 318, 709, 715, 333, 848, 209, 598, 726, 728, 343, 349, 617, 362, 366, 381, 893}
Object name: chair | Missing ids: {579, 580}
Object name: cushion | Missing ids: {529, 495, 542, 583}
Object name: stool | Missing ids: {561}
Object name: table | Missing ids: {481, 564, 517, 582}
Object name: tv_monitor | Missing ids: {329, 538, 150, 151, 154}
Object name: plant | Missing ids: {194, 514, 328, 141, 718, 721, 694}
Object name: sofa | Missing ids: {574}
Object name: fireplace | Missing ids: {639}

Here's the gist to reproduce these results: https://gist.github.com/srama2512/9a82995a2ab9c40ba9721747c86fe325

I used a relatively older habitat-lab commit from git (May 2020): e0807ed403902de78ee37b36766c2f06e5886fea But I don't think this should affect the analysis much (except for a few changes in make_habitat_configuration).

[mathfac]

I will take a closer look. One of the reasons can be that there were no view points around that goal objects that satisfies the success criteria (object can be seen from 1m distance to bounding box).

On Wed, Feb 3, 2021 at 9:43 PM Santhosh Kumar Ramakrishnan < notifications@github.com> wrote:

❓ Questions and Help

I noticed that objectnav agents do not get rewarded for visiting certain objects from the target category. I wanted to investigate this further, and see if there's a difference b/w the set of objects present in the objectnav task dataset, and the set of objects in the scene.

Here are a set of object categories from scene 1pXnuDYAj8r in MP3D. All these objects have a set of instances that are missing from the objectnav dataset.

Object name: cabinet | Missing ids: {875, 430, 432, 434, 595}

Object name: bathtub | Missing ids: {207}

Object name: towel | Missing ids: {269, 942, 175, 213, 184, 30}

Object name: picture | Missing ids: {512, 897, 142, 143, 785, 146, 275, 786, 787, 788, 155, 159, 671, 289, 290, 291, 804, 550, 556, 301, 820, 181, 826, 699, 315, 955, 318, 709, 715, 333, 848, 209, 598, 726, 728, 343, 349, 617, 362, 366, 381, 893}

Object name: chair | Missing ids: {579, 580}

Object name: cushion | Missing ids: {529, 495, 542, 583}

Object name: stool | Missing ids: {561}

Object name: table | Missing ids: {481, 564, 517, 582}

Object name: tv_monitor | Missing ids: {329, 538, 150, 151, 154}

Object name: plant | Missing ids: {194, 514, 328, 141, 718, 721, 694}

Object name: sofa | Missing ids: {574}

Object name: fireplace | Missing ids: {639}

Here's the gist to reproduce these results: https://gist.github.com/srama2512/9a82995a2ab9c40ba9721747c86fe325

I used a relatively older habitat-lab commit from git (May 2020): e0807ed https://github.com/facebookresearch/habitat-lab/commit/e0807ed403902de78ee37b36766c2f06e5886fea But I don't think this should affect the analysis much (except for a few changes in make_habitat_configuration).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/facebookresearch/habitat-lab/issues/583, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGWHACWUULEUR5EJJIFT5TS5IXYPANCNFSM4XCFWYOA .

[srama2512]

Got it. I wasn't sure how to eliminate those cases here. I do remember seeing some qualitative examples where this happens. I don't have one that I can provide right now though. Please let me know if you need more details on such cases.