search

allenai / procthor-rl

RL training scripts for learning an agent using ProcTHOR.
Apache License 2.0
14 stars 3 forks source link

zero-shot object navigation #2

Open leyuan-sun opened 6 months ago

leyuan-sun commented 6 months ago

I saw the evaluation of object navigation on ARCHITECTHOR/AI2-iTHOR/RoboTHOR, what is the definition of zero-shot in your evaluation, seen object classes for training and unseen classes for evaluation? Could you plz give more explantions of evaluations setting and how to reproduce the results in this repo, thanks in advance!!

KuoHaoZeng commented 6 months ago

Hello! The zero-shot means the policy model only sees the ProcTHOR-10k training scene during the training stage, and the scenes designed by our in-house designers, including ArchitecTHOR/iTHOR, were not accessible at training. In addition, RoboTHOR was also not seen at training, and the RoboTHOR scene looks quite different from ProcTHOR-10k because of its unique texture and layout.

leyuan-sun commented 6 months ago

Thanks for your reply! Is it possible to check the difference of target object classes between training and evaluation, since I would like to do more traditional zero shot object goal navigation evaluation, which is seen target classes in training and unseen object target classes in evaluation.