Initializing a soundspaces environment

[sreeharshaparuchur1]

Hello there @ChanganVR,

I would like to create an 'env' object similar to the one defined here that has a 'step' function and returns the observations, rewards, dones and infos for every timestep, initially on just one environment (unlike the multiple that the code snipped referenced above can handle).

So how do I go about doing this? I've tried tracing the code for multiple models that use soundspaces such as av_wan, av_nav, savi and Move2Hear. However, these codes initialize a 'Trainer' from the baseline registry class which I am unable to breakdown and trace to locate where the env class is being initialized.

The method of initializing a soundspaces environment given in the soundspaces2_quick_tutorial won't work for my desired application as it requires very low-level initialization of the scene_dataset, camera parameters and agent action space.

So, I have tried initializing the environment similar to how it's done in the shortest path follower example script. As I want to initialize an AudioNav environment, I have modified the code and config loaded as follows:

the code:

from habitat.datasets import make_dataset
from ss_baselines.av_nav.config import get_config
from ss_baselines.common.environments import AudioNavRLEnv

config = get_config(
    config_paths="/home/lordperceval/Perception/sound-spaces/configs/audionav/av_nav/mp3d/generalized_soundspaces_audiogoal_config.yaml",
    run_type="eval")

dataset = make_dataset(id_dataset=config.DATASET.TYPE, config=config.DATASET)

env = AudioNavRLEnv(config = config, dataset=dataset)

the config loaded above:

ENVIRONMENT:
  MAX_EPISODE_STEPS: 500
SIMULATOR:
  TURN_ANGLE: 90
  FORWARD_STEP_SIZE: 1.0
  HABITAT_SIM_V0:
    GPU_DEVICE_ID: 0
  RGB_SENSOR:
    WIDTH: 128
    HEIGHT: 128
  DEPTH_SENSOR:
    WIDTH: 128
    HEIGHT: 128

  TYPE: "SoundSpacesSim"
  ACTION_SPACE_CONFIG: "v0"
  SCENE_DATASET: "mp3d"
  GRID_SIZE: 1.0
  USE_RENDERED_OBSERVATIONS: False #True
  CONTINUOUS_VIEW_CHANGE: False
  VIEW_CHANGE_FPS: 30
  AUDIO:
    RIR_SAMPLING_RATE: 16000
    BINAURAL_RIR_DIR: "/media/lordperceval/Seagate Expansion Drive/binaural_rirs"
    SOURCE_SOUND_DIR: "/home/lordperceval/Perception/sound-spaces/data/sounds/1s_all"
    HAS_DISTRACTOR_SOUND: False

TASK:
  TYPE: AudioNav
  SUCCESS_DISTANCE: 1.0

  SENSORS: ['AUDIOGOAL_SENSOR']
  GOAL_SENSOR_UUID: audiogoal

  MEASUREMENTS: ['DISTANCE_TO_GOAL', 'NORMALIZED_DISTANCE_TO_GOAL', 'SUCCESS', 'SPL', 'SOFT_SPL']
  SUCCESS:
    SUCCESS_DISTANCE: 1.0
  TOP_DOWN_MAP:
    MAP_RESOLUTION: 500
    DRAW_SOURCE_AND_TARGET: True
    DRAW_BORDER: True
    DRAW_SHORTEST_PATH: True

DATASET:
  TYPE: "AudioNav"
  SPLIT: "val_telephone"
  CONTENT_SCENES: ["*"]
  VERSION: 'v1'
  # SCENES_DIR: "/home/lordperceval/Perception/sound-spaces/data/metadata/mp3d"
  SCENES_DIR: "/home/lordperceval/Perception/sound-spaces/data/scene_datasets/mp3d"
  DATA_PATH: "/home/lordperceval/Datasets/decision_transformer_false_walk_json/val_farTarget_8scenes_100episodes/False_Walk_02/val_farTarget_8scenes_100episodes.json.gz"

However, when I try to initialize a greedyFollower as given in the shortest_path_follower_example with:

follower = GreedyGeodesicFollower(env.habitat_env.sim, 1e-05, False)

I run into this error:

Traceback (most recent call last):
  File "test_env_1.py", line 39, in <module>
    follower = GreedyGeodesicFollower(env.habitat_env.sim, 1e-05, False)
  File "/home/lordperceval/anaconda3/envs/habitat22testing/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/nav/greedy_geodesic_follower.py", line 87, in __init__
    key, spec = self._find_action("move_forward")
  File "/home/lordperceval/anaconda3/envs/habitat22testing/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/nav/greedy_geodesic_follower.py", line 125, in _find_action
    self.agent.agent_config.action_space.items(),
AttributeError: 'float' object has no attribute 'agent_config'

what's even more concerning is that on printing the agent characteristics given by print(f"{env.habitat_env.sim.agents}")

I see this:

2023-07-11 17:55:58,208 Initializing task Nav-v0
[Agent(agent_config=AgentConfiguration(height=1.5, radius=0.1, sensor_specifications=[<habitat_sim._ext.habitat_sim_bindings.CameraSensorSpec object at 0x7f62811e4b90>], action_space={0: ActionSpec(name='stop', actuation=None), 1: ActionSpec(name='move_forward', actuation=ActuationSpec(amount=0.5, constraint=None)), 2: ActionSpec(name='turn_left', actuation=ActuationSpec(amount=10, constraint=None)), 3: ActionSpec(name='turn_right', actuation=ActuationSpec(amount=10, constraint=None))}, body_type='cylinder'), _sensors={'rgb': <habitat_sim._ext.habitat_sim_bindings.CameraSensor object at 0x7f629a601470>}, controls=ObjectControls(move_filter_fn=<bound method Simulator.step_filter of HabitatSim(config=Configuration(sim_cfg=<habitat_sim._ext.habitat_sim_bindings.SimulatorConfiguration object at 0x7f629a547230>, agents=[AgentConfiguration(height=1.5, radius=0.1, sensor_specifications=[<habitat_sim._ext.habitat_sim_bindings.CameraSensorSpec object at 0x7f62811e4b90>], action_space={0: ActionSpec(name='stop', actuation=None), 1: ActionSpec(name='move_forward', actuation=ActuationSpec(amount=0.5, constraint=None)), 2: ActionSpec(name='turn_left', actuation=ActuationSpec(amount=10, constraint=None)), 3: ActionSpec(name='turn_right', actuation=ActuationSpec(amount=10, constraint=None))}, body_type='cylinder')], metadata_mediator=None), agents=[...], _num_total_frames=0, _default_agent_id=0, _Simulator__sensors=[{'rgb': <habitat_sim.simulator.Sensor object at 0x7f629a51ad10>}], _initialized=True, _previous_step_time=0.0, _async_draw_agent_ids=None, _Simulator__last_state={0: AgentState(position=array([-4.768345 , -2.8503408, -1.3496051], dtype=float32), rotation=quaternion(1, 0, 2.45858027483337e-05, 0), sensor_states={'rgb': SixDOFPose(position=array([-4.768345 , -1.6003408, -1.3496051], dtype=float32), rotation=quaternion(1, 0, 2.45858027483337e-05, 0))})})>), body=<_magnum.scenegraph.AbstractFeature3D object at 0x7f629a53b8f0>)]

This is not expected behaviour as the ActuationSpec object above does not reflect the manually set TURN_ANGLE and FORWARD_STEP_SIZE from the above config.

Kindly let me know what I'm doing wrong and what a simple way to get the class above would be as well as how I can manually adjust the rewards for each timestamp as I would want to design a reward of my own and test it in the soundspaces environment.

Thank you!

[ChanganVR]

Hi @sreeharshaparuchur1 sorry for the late reply. Have you figured out how to create this function? If not, since you have several questions, feel free to reach out to me by email to schedule a video call so I can answer your questions altogether and help you debug.