Joel Ye, Dhruv Batra, Abhishek Das, and Erik Wijmans.
Project Site: https://joel99.github.io/objectnav/
This repo is a code supplement for the paper "Auxiliary Tasks and Exploration Enable ObjectNav." The code is a fork of Habitat-Lab and borrows heavily from habitat-pointnav-aux.
The primary code contributions from the paper are located in:
auxiliary_tasks.py
and supervised_auxiliary_tasks.py
coverage.py
rollout_storage.py
, ppo.py
, multipolicy.py
habitat_baselines/config/objectnav/full/*.yaml
ppo_trainer.py
./scripts/*
The code is known to be compatible with Python 3.6, Habitat Lab v0.1.5 and Habitat Sim v0.1.6 (headless). We recommend you use conda to initialize the code as follows:
conda create -n <ENV_NAME> python=3.6 cmake=3.14.0
conda activate <ENV_NAME>
pip install -r requirements.txt
conda install habitat-sim=0.1.6 headless -c conda-forge -c aihabitat
Then, in this repo, run:
conda env update -f environment.yml --name <ENV_NAME>
Note: If you run into a pip error along the lines of a missing metadata file, it is likely an improperly cleaned package (e.g. numpy-1.19.5), just remove it.
Note 2: You may also need to run python setup.py develop --all
as in the habitat-lab installation process.
The experiments were done on the Matterport3D dataset. Please refer to habitat-sim for the scene dataset and habitat-lab for the ObjectNav dataset.
You can download pretrained agent and rednet weights here. We provide the best checkpoint for agents as measured by their performance with RedNet segmentation:
base-full.31.pth
: 6-action agent.base4-full.33.pth
: 4-action agent.split_clamp.31.pth
: 6 + Tether agent.rednet_semmap_mp3d_tuned.pth
: RedNet weights.We train our models for around 8 GPU-weeks, on a SLURM-managed cluster. On such a cluster, you can launch your run with:
sbat ./scripts/ddppo_8gpu.sh <args>
With the appropriate configuration file. The code in run.py
is set up to parse out the arguments, so to train with the configuration .../objectnav/full/base.yaml
, write:
sbat ./scripts/ddppo_8gpu.sh base full
On a machine without SLURM, see, e.g. ./scripts/train_no_ckpt.sh
.
You can evaluate a trained checkpoint by configuring the arguments in ./scripts/eval_on.sh
. This forwards to habitat_baselines/run.py
, which accepts a number of flags. For example:
-v
flag to generate videos of your agent.-ra
flag to record trajectories and hidden states for subsequent analysis.See run.py
for all arguments. Note that an evaluation on the full validation split can take a while (>1 GPU-day).
If you use this work, you can cite it as
@misc{ye2021auxiliary,
title={Auxiliary Tasks and Exploration Enable ObjectNav},
author={Joel Ye and Dhruv Batra and Abhishek Das and Erik Wijmans},
year={2021},
eprint={2104.04112},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
scripts/dataset_slicer.py
.py
notebooks in scripts/
start with a comment denoting their use.