search

facebookresearch / sound-spaces

A first-of-its-kind acoustic simulation platform for audio-visual embodied AI research. It supports training and evaluating multiple tasks and applications.
https://soundspaces.org
Creative Commons Attribution 4.0 International
345 stars 55 forks source link

AttributeError: TASK on using SoundSpaces 2.0 #100

Open sreeharshaparuchur1 opened 1 year ago

sreeharshaparuchur1 commented 1 year ago

Hi,

In my Soundspaces 2.0 environment ,that was set up following the instructions given in the readme, I am unable to successfully execute the following commands (taken from here):

python scripts/interactive_demo.py

And

python ss_baselines/av_nav/run.py --exp-config ss_baselines/av_nav/config/audionav/mp3d/train_telephone/audiogoal_depth_ddppo.yaml --model-dir data/models/ss2/mp3d/dav_nav CONTINUOUS True

The error log on attempting to execute both of the commands above is the same - 

pygame 2.1.2 (SDL 2.0.16, Python 3.7.13)
Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
  File "scripts/interactive_demo.py", line 18, in <module>
    import soundspaces
  File "/home/chieftwit/SoundSpaces20/sound-spaces/soundspaces/__init__.py", line 17, in <module>
    from soundspaces.challenge import Challenge
  File "/home/chieftwit/SoundSpaces20/sound-spaces/soundspaces/challenge.py", line 12, in <module>
    from soundspaces.benchmark import Benchmark
  File "/home/chieftwit/SoundSpaces20/sound-spaces/soundspaces/benchmark.py", line 21, in <module>
    from ss_baselines.av_nav.config import get_task_config
  File "/home/chieftwit/SoundSpaces20/sound-spaces/ss_baselines/av_nav/config/__init__.py", line 1, in <module>
    from ss_baselines.av_nav.config.default import get_task_config, get_config
  File "/home/chieftwit/SoundSpaces20/sound-spaces/ss_baselines/av_nav/config/default.py", line 100, in <module>
    _TC.TASK.AUDIOGOAL_SENSOR = CN()
  File "/home/chieftwit/anaconda3/envs/ss20/lib/python3.7/site-packages/yacs/config.py", line 141, in __getattr__
    raise AttributeError(name)
AttributeError: TASK

(The referenced comment speaks about interactive_mode.py, I'm assuming it's a typo and the author meant interactive_demo.py)

I have tried digging into the codebase and figuring out why I'm running into this error as I have not touched any of the files from the repo. Most of the code in the corresponding files (default.py, the config yaml files) are similar to their corresponding files in my Soundspaces 1.0 environment save for the BINAURAL_RIR_DIR parameter related code.

Based on the above, I conjecture that the error may be due to an API issue (not sure where though).

Since this error is at the task config level, I figured that none of the hacks that were proposed to get Soundspaces 1.0's interactive demo working would be applicable.

Kindly help me out.

Lastly, sorry about the 100th issue for this spectacular project being as lame as this but I can't figure it out :(

ChanganVR commented 1 year ago

@sreeharshaparuchur1 Looks like you were using the incorrect version of Habitat-Lab and thus there is a conflict of APIs. Which version of Habitat-Lab are you using? You can fix that easily by switching to correct version.

Unfortunately I can't get to the code at the moment, but if you let me know which version you installed. I'll try to re-produce it next week!

sreeharshaparuchur1 commented 1 year ago

Hi @ChanganVR ,

Apologies, the above error was due to an API conflict that came up on using the wrong habitat-lab version.

However, on installing the right version, I have faced several issues in getting the interactive demo for soundspaces 2.0 to work. I have tried to get to the root of the problem for several days now but without much luck.

On running python scripts/interactive_demo.py CONTINUOUS True (as I want to run an interactive demo using the SS20 simulator), I run into the following error:

[12:00:57:006659]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 00:Flat.
... 

[12:00:57:110690]:[Sim] Simulator.cpp(439)::instanceStageForSceneAttributes : Successfully loaded stage named : data/scene_datasets/mp3d/QUCTc6BB5sX/QUCTc6BB5sX.glb
[12:00:57:110796]:[Sim] Simulator.cpp(205)::reconfigure : CreateSceneInstance success == true for active scene name : data/scene_datasets/mp3d/QUCTc6BB5sX/QUCTc6BB5sX.glb  with renderer.
[12:00:57:113008]:[Sensor] AudioSensor.cpp(24)::AudioSensorSpec : [Audio]  AudioSensorSpec constructor
[12:00:57:113094]:[Sensor] AudioSensor.cpp(42)::sanityCheck : [Audio]  SanityCheck the audio sensor spec
[12:00:57:113111]:[Sensor] AudioSensor.cpp(56)::AudioSensor : [Audio]  AudioSensor constructor
[12:00:57:113122]:[Sensor] AudioSensor.cpp(42)::sanityCheck : [Audio]  SanityCheck the audio sensor spec
[12:00:57:113167]:[Sensor] AudioSensor.cpp(169)::setAudioMaterialsJSON : Set audio materials database to json file :  /home/bwoah/SoundSpaces20/sound-spaces/data/mp3d_material_config.json
2022-11-24 12:00:57,113 Initializing task AudioNav
[12:00:58:164753]:[Sensor] AudioSensor.cpp(86)::setAudioSourceTransform : [Audio]  Setting the audio source position :  Vector(-6.43414, -1.46512, -21.2952) ]
[12:00:58:164834]:[Sensor] AudioSensor.cpp(321)::createAudioSimulator : [Audio]  Create audio simulator
CreateContext: Context created
Fatal Python error: (pygame parachute) Segmentation Fault

Current thread 0x00007f326e660280 (most recent call first):
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/simulator.py", line 663 in draw_observation
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/simulator.py", line 436 in get_sensor_observations
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/simulator.py", line 185 in reset
  File "/home/bwoah/SoundSpaces20/sound-spaces/soundspaces/continuous_simulator.py", line 372 in reset
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/core/embodied_task.py", line 284 in reset
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/core/env.py", line 252 in reset
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/core/env.py", line 402 in reset
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/contextlib.py", line 74 in inner
  File "/home/bwoah/SoundSpaces20/sound-spaces/ss_baselines/common/environments.py", line 53 in reset
  File "scripts/interactive_demo.py", line 52 in interactive_demo
  File "scripts/interactive_demo.py", line 238 in main
  File "scripts/interactive_demo.py", line 245 in <module>
Aborted (core dumped)

Issue #44 in this repo has a similar error log, the changes mentioned in that issue did enable me to run the interactive_demo for soundspaces 1.0 successfully. With the exception of setting USE_RENDERED_OBSERVATIONS True, I have made the changes to the script but face the above issue.

On digging a little bit, I commented out these lines in the interactive demo and added env.reset() after this line.

My log changed considerably as in addition to seeing the above ResourceManager.cpp calls, I see

[12:12:49:895057]:[Sensor] AudioSensor.cpp(104)::setAudioListenerTransform : [Audio]  Setting the agent transform : position [ Vector(-3.44611, -1.62041, -18.2299) ], rotQuat[ Vector(1, 0, 2.45858e-05, 0) ]
[12:12:49:895116]:[Sensor] AudioSensor.cpp(321)::createAudioSimulator : [Audio]  Create audio simulator
[12:12:49:895131]:[Sensor] AudioSensor.cpp(130)::runSimulation : [Audio]  Running the audio simulator
[12:12:49:895139]:[Sensor] AudioSensor.cpp(135)::runSimulation : [Audio]  New initialization, will upload geometry
[12:12:49:895147]:[Sensor] AudioSensor.cpp(139)::runSimulation : [Audio]  Loading semantic scene
[12:12:49:895154]:[Sensor] AudioSensor.cpp(361)::loadSemanticMesh : [Audio]  Loading semantic mesh true
[12:12:52:355708]:[Sensor] AudioSensor.cpp(450)::loadSemanticMesh : [Audio]   Index count:  149721 , Material:  shower
...

[12:12:52:495867]:[Sensor] AudioSensor.cpp(450)::loadSemanticMesh : [Audio]   Index count:  886272 , Material:  table
[12:12:52:497596]:[Sensor] AudioSensor.cpp(450)::loadSemanticMesh : [Audio]   Index count:  182271 , Material:  stool
[12:15:28:461636]:[Sensor] AudioSensor.cpp(104)::setAudioListenerTransform : [Audio]  Setting the agent transform : position [ Vector(0.565855, -1.56512, -15.2952) ], rotQuat[ Vector(1, 0, 0, 0) ]
[12:15:28:461704]:[Sensor] AudioSensor.cpp(321)::createAudioSimulator : [Audio]  Create audio simulator
[12:15:28:461722]:[Sensor] AudioSensor.cpp(130)::runSimulation : [Audio]  Running the audio simulator

The interactive demo display pops up and doesn't crash immediately. But on pressing a key (D), I get a segmentation fault with the following error message:

Fatal Python error: (pygame parachute) Segmentation Fault

Current thread 0x00007f0f82952280 (most recent call first):
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/simulator.py", line 663 in draw_observation
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/simulator.py", line 436 in get_sensor_observations
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/site-packages/habitat_sim-0.2.2-py3.7-linux-x86_64.egg/habitat_sim/simulator.py", line 506 in step
  File "/home/bwoah/SoundSpaces20/sound-spaces/soundspaces/continuous_simulator.py", line 402 in step
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/tasks/nav/nav.py", line 1026 in step
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/core/embodied_task.py", line 316 in _step_single_action
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/core/embodied_task.py", line 336 in step
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/core/env.py", line 300 in step
  File "/home/bwoah/SoundSpaces20/habitat-lab-0.2.2/habitat/core/env.py", line 447 in step
  File "/home/bwoah/anaconda3/envs/ss20/lib/python3.7/contextlib.py", line 74 in inner
  File "/home/bwoah/SoundSpaces20/sound-spaces/ss_baselines/common/environments.py", line 66 in step
  File "scripts/interactive_demo.py", line 86 in interactive_demo
  File "scripts/interactive_demo.py", line 238 in main
  File "scripts/interactive_demo.py", line 245 in <module>
Aborted (core dumped)

I have tried to debug the issue for a while and while it makes sense to call env.reset() before initializing a new scene, I am unable to understand why sim.reset() is called at every step and why it is causing a segmentation fault. It would be highly appreciated if you could give some insight into when and why the above calls are made.

Thanks in advance!