search

facebookresearch / sound-spaces

A first-of-its-kind acoustic simulation platform for audio-visual embodied AI research. It supports training and evaluating multiple tasks and applications.
https://soundspaces.org
Creative Commons Attribution 4.0 International
322 stars 55 forks source link

How to use ShortestPathFollower ? #114

Closed dosssman closed 1 year ago

dosssman commented 1 year ago

Hello there.

Long time no see.

While going through the sound-spaces files, I could not help but notice that there is a ShortestPathFollower class here.

Would it be possible to use this class to instantiate as a form of oracle to generate optimal trajectories across the scenes ? Also, is it perhaps compatible with SAVi environments ?

Thank you for your time.

ChanganVR commented 1 year ago

Hi @dosssman, yes, you can use this to generate the oracle trajectory for either AudioGoal or Semantic AudioGoal, although it uses information that is not accessible by a real navigation agent.

dosssman commented 1 year ago

Hello again.

Thank you very much for taking the time to check on this issues.

Just leaving this here for other people that might stumble upon this issue:

When I was trying to use that class, it would fail instantiating as done in the ppo_trainer.py, it would throw and error along the lines of DummySimulator does not have '_path_finder' attribute.

Workaround to get it working somehow, was to not use pre-rendered observations, by having config.TASK_CONFIG.SIMULATOR.USE_RENDERED_OBSERVATIONS = False.

Another alternative I found to get optimal actions for an episode was to access envs.workers[0]._env._env._sim._oracle_actions, which returns a list of actions that lead the agent to the goal location upon execution. This method could even be used with USE_RENDERED_OBSERVATIONS set to True.

This was tested with the SAVI environment, whcih uses MP3D dataset. Not sure about Replica.

Thanks again for your time. I would consider this issue closed.