facebookresearch / sound-spaces

A first-of-its-kind acoustic simulation platform for audio-visual embodied AI research. It supports training and evaluating multiple tasks and applications.
https://soundspaces.org
Creative Commons Attribution 4.0 International
364 stars 58 forks source link

Wired Behavior of AudioSensor #127

Open ZetaRing opened 1 year ago

ZetaRing commented 1 year ago

Hi, I am working with sound space but found something strange. Can anyone help me out? I am reading RIR from a audio sensor but the length of RIR arrays changed after first time. I think the RIR array should be same because the agent did not move and the simulator time did not change. Here is my code:

    cfg = conf()
    sim = habitat_sim.Simulator(cfg)

    audio_sensor = sim.get_agent(0)._sensors["audio_sensor"]
    audio_sensor.setAudioMaterialsJSON("data/mp3d_material_config.json")
    audio_sensor.setAudioSourceTransform(np.array(source))
    for i in range(4):
        obs = sim.get_sensor_observations()
        rir = np.array(obs["audio_sensor"]).T
        print(sim.get_world_time(), len(rir))

Here is the output:

0.0 17194
0.0 19065
0.0 19065
0.0 19065

Here is the config of the audio sensor:

    audio_sensor_spec = habitat_sim.AudioSensorSpec()
    audio_sensor_spec.uuid = "audio_sensor"
    audio_sensor_spec.enableMaterials = True  # make sure _semantic.ply file is in the scene folder
    audio_sensor_spec.channelLayout.type = habitat_sim.sensor.RLRAudioPropagationChannelLayoutType.Mono
    audio_sensor_spec.channelLayout.channelCount = 1
    audio_sensor_spec.position = [0.0, 1.5, 0.0]
    audio_sensor_spec.acousticsConfig.sampleRate = 16000
    audio_sensor_spec.acousticsConfig.indirect = True
    sensors.append(audio_sensor_spec)
ChanganVR commented 1 year ago

@Wind-Wing the acoustics simulation is based on Monte-Carlo path tracing to speed things up so there is a little bit of stochasticity (expected). However, the macro characteristics of the impulse response should be almost the same, e.g., rt60. If you plot these impulse responses, you should not really see a difference. Let me know if this is what you observe.

ZetaRing commented 1 year ago

Thanks for the replay! The plots are almost the same. However, I think the doc should mention this problem. This output is really confusing.

ChanganVR commented 1 year ago

Sounds good. Will add that.