facebookresearch / sound-spaces

A first-of-its-kind acoustic simulation platform for audio-visual embodied AI research. It supports training and evaluating multiple tasks and applications.
https://soundspaces.org
Creative Commons Attribution 4.0 International
322 stars 55 forks source link

Wired Behavior of AudioSensor #127

Open ZetaRing opened 10 months ago

ZetaRing commented 10 months ago

Hi, I am working with sound space but found something strange. Can anyone help me out? I am reading RIR from a audio sensor but the length of RIR arrays changed after first time. I think the RIR array should be same because the agent did not move and the simulator time did not change. Here is my code:

    cfg = conf()
    sim = habitat_sim.Simulator(cfg)

    audio_sensor = sim.get_agent(0)._sensors["audio_sensor"]
    audio_sensor.setAudioMaterialsJSON("data/mp3d_material_config.json")
    audio_sensor.setAudioSourceTransform(np.array(source))
    for i in range(4):
        obs = sim.get_sensor_observations()
        rir = np.array(obs["audio_sensor"]).T
        print(sim.get_world_time(), len(rir))

Here is the output:

0.0 17194
0.0 19065
0.0 19065
0.0 19065

Here is the config of the audio sensor:

    audio_sensor_spec = habitat_sim.AudioSensorSpec()
    audio_sensor_spec.uuid = "audio_sensor"
    audio_sensor_spec.enableMaterials = True  # make sure _semantic.ply file is in the scene folder
    audio_sensor_spec.channelLayout.type = habitat_sim.sensor.RLRAudioPropagationChannelLayoutType.Mono
    audio_sensor_spec.channelLayout.channelCount = 1
    audio_sensor_spec.position = [0.0, 1.5, 0.0]
    audio_sensor_spec.acousticsConfig.sampleRate = 16000
    audio_sensor_spec.acousticsConfig.indirect = True
    sensors.append(audio_sensor_spec)
ChanganVR commented 9 months ago

@Wind-Wing the acoustics simulation is based on Monte-Carlo path tracing to speed things up so there is a little bit of stochasticity (expected). However, the macro characteristics of the impulse response should be almost the same, e.g., rt60. If you plot these impulse responses, you should not really see a difference. Let me know if this is what you observe.

ZetaRing commented 9 months ago

Thanks for the replay! The plots are almost the same. However, I think the doc should mention this problem. This output is really confusing.

ChanganVR commented 9 months ago

Sounds good. Will add that.