IIT-PAVIS / Look_Around_And_Learn

Code for the paper titled "Look Around šŸ‘€ and Learn šŸŽ“: Self-Training Object Detection by Exploration" accepted at ECCV2024
MIT License
9 stars 0 forks source link

Look around and learn

Setup

Data

All the data are contained inside the data directory.

Example: habitat test scenes

  1. Download test-scenes http://dl.fbaipublicfiles.com/habitat/habitat-test-scenes.zip
  2. Unzip inside project directory

Launch an experiment

python scripts/run_exp.py run training or deployment of a policy. More information about "RL baselines" and our RL policy below.

Replay experiment

To replay an experiment, use the following python scripts/visualize_exp.py replay.episode_id={ID episode} replay.exp_name={PATH TO EPISODE} replay.modalities="['rgb', 'depth','semantic']"

Run baselines

The following learned baselines are implemented:

The following classical baselines are implemented:

Train goalexploration policy

Start from confs/habitat/gibson_goal_exploration.yaml

Environments:

Environments for the RL baselines are also provided:

Policies:

Checkpoints:

Generate from goalexploration policy

Start from confs/habitat/gibson_goal_exploration.yaml

Datasets

Scenes datasets

Scenes models Extract path Archive size
Gibson data/scene_datasets/gibson/{scene}.glb 1.5 GB
MatterPort3D data/scene_datasets/mp3d/{scene}/{scene}.glb 15 GB

Tasks

Gibson object navigation (used for the main results of the paper)

You can download the task at the following link {ADD LINK}, unzip and put it in data/datasets/objectnav/gibson/v1.1

Habitat's pointnav

Task Scenes Link Extract path Config to use Archive size
Point goal navigation Gibson pointnav_gibson_v1.zip data/datasets/pointnav/gibson/v1/ datasets/pointnav/gibson.yaml 385 MB
Point goal navigation corresponding to Sim2LoCoBot experiment configuration Gibson pointnav_gibson_v2.zip data/datasets/pointnav/gibson/v2/ datasets/pointnav/gibson_v2.yaml 274 MB
Point goal navigation MatterPort3D pointnav_mp3d_v1.zip data/datasets/pointnav/mp3d/v1/ datasets/pointnav/mp3d.yaml 400 MB

MatterPort

Gibson

Dependencies

Contributing

If you want to contribute to the project, I suggest to install requirements-dev.txt and abilitate pre-commit

python -m pip install -r requirements-dev.txt
pre-commit install