search

facebookresearch / habitat-lab

A modular high-level library to train embodied AI agents across a variety of tasks and environments.
https://aihabitat.org/
MIT License
1.96k stars 492 forks source link

Question about the pretrained RGBD PPO model for Gibson #1008

Closed yingchengyang closed 1 year ago

yingchengyang commented 1 year ago

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: master

Habitat-Sim: master

❓ Questions and Help

Thanks a lot for designing such an excellent codebase for Embodied AI! I have some questions about the performance of pretrained models provided in habitat_baselines.

I have tested trained RGB, RGBD, and Depth PPO models for Gibson 100 trajectories. The success rate of the RGB model and the Depth model is 0.77 and 0.90 respectively, which are extremely great. Unfortunately, the success rate of the RGBD model is only 0.15. Does the use of the RGBD model require some special treatment?

Thank you!

dhruvbatra commented 1 year ago

I suspect you're using the models from here: https://github.com/facebookresearch/habitat-lab/tree/main/habitat-baselines/habitat_baselines

That's part of our 2019 ICCV release. At that time, we did find RGBD models performing poorly, but the code has changed a lot since then so it's hard to be confident why you're seeing what you're seeing.

I would recommend using the models from the DD-PPO paper here: https://github.com/facebookresearch/habitat-lab/tree/main/habitat-baselines/habitat_baselines/rl/ddppo

Or even better our most recent models from the HM3D paper here: https://github.com/facebookresearch/habitat-matterport3d-dataset/tree/main/pointnav_comparison