search

facebookresearch / sound-spaces

A first-of-its-kind acoustic simulation platform for audio-visual embodied AI research. It supports training and evaluating multiple tasks and applications.
https://soundspaces.org
Creative Commons Attribution 4.0 International
364 stars 58 forks source link

Multiple sound sources and goal locations #116

Closed kksinghal closed 1 year ago

kksinghal commented 1 year ago

Hello, I want to create a custom environment where I want to add multiple sound sources (with different sounds) and set those as goals. But I couldn't find the code that initialises the sound source position and sets that position as a goal. Also, what changes will I need to make to do that?

Thanks Kartik

ChanganVR commented 1 year ago

For SS 1, it's essentially setting the index for source/reciever and retrieving the corresponding RIR, for which, you can check out the function here: https://github.com/facebookresearch/sound-spaces/blob/3768a5073fb305723676b07cb2a78b62f0d7f6c6/soundspaces/simulator.py#L478

For SS 2 (on the fly rendering), you can check out the tutorial here: https://github.com/facebookresearch/sound-spaces/blob/main/examples/soundspaces2_quick_tutorial.ipynb

kksinghal commented 1 year ago

@ChanganVR Thanks for the info. Sorry, I was struggling with installation earlier, so I couldn't respond before. But, they are solved.

I would like to change the current code for my purpose. I changed soundspaces/datasets/audionav_dataset.py and added two goal positions https://github.com/facebookresearch/sound-spaces/blob/3768a5073fb305723676b07cb2a78b62f0d7f6c6/soundspaces/datasets/audionav_dataset.py#L153 And I also checked in the environment object that it shows two goals. However, in the terminal log, it seems that only one goal (first one) is being used to initialise the sound source, and I also receive the observations (based on the GPS compass) according to that one goal only. I suppose that habitat supports only 1 goal scenario (https://github.com/facebookresearch/habitat-lab/issues/735#issuecomment-929389560), but I will be able to solve the purpose if I can add the sound sources at the multiple goal locations, then I will write the metrics code myself.

Are there any other changes that are required?

@dosssman Your issue discussions with Changan have been very useful. Please share your knowledge as well. Thanks All.

LOGS: I see that setAudioSourceTransform is logged only once. I have highlighted some lines, which shows some of my print statements.

pygame 2.0.1 (SDL 2.0.14, Python 3.9.16) Hello from the pygame community. https://www.pygame.org/contribute.html 2023-05-25 16:55:43.930251: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-05-25 16:55:44.513762: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2023-05-25 16:55:45,823 Initializing dataset AudioNav 2023-05-25 16:55:45, INFO: Sampled 1 from 17DRP5sb8fy 2023-05-25 16:55:45,830 initializing sim ContinuousSoundSpacesSim [16:55:45:833109]:[Metadata] AttributesManagerBase.h(380)::createFromJsonOrDefaultInternal : : Proposing JSON name : default.scene_dataset_config.json from original name : default| This file does not exist. [16:55:45:833205]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DSolid:capsule3DSolid_hemiRings_4_cylRings_1_segments_12_halfLen_0.75_useTexCoords_false_useTangents_false) created and registered. [16:55:45:833233]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DWireframe:capsule3DWireframe_hemiRings_8_cylRings_1_segments_16_halfLen_1) created and registered. [16:55:45:833252]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneSolid:coneSolid_segments_12_halfLen_1.25_rings_1_useTexCoords_false_useTangents_false_capEnd_true) created and registered. [16:55:45:833269]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneWireframe:coneWireframe_segments_32_halfLen_1.25) created and registered. [16:55:45:833283]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeSolid:cubeSolid) created and registered. [16:55:45:833296]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeWireframe:cubeWireframe) created and registered. [16:55:45:833312]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderSolid:cylinderSolid_rings_1_segments_12_halfLen_1_useTexCoords_false_useTangents_false_capEnds_true) created and registered. [16:55:45:833328]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderWireframe:cylinderWireframe_rings_1_segments_32_halfLen_1) created and registered. [16:55:45:833342]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereSolid:icosphereSolid_subdivs_1) created and registered. [16:55:45:833357]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereWireframe:icosphereWireframe_subdivs_1) created and registered. [16:55:45:833371]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereSolid:uvSphereSolid_rings_8_segments_16_useTexCoords_false_useTangents_false) created and registered. [16:55:45:833386]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereWireframe:uvSphereWireframe_rings_16_segments_32) created and registered. [16:55:45:833396]:[Metadata] AssetAttributesManager.cpp(110)::AssetAttributesManager : Built default primitive asset templates : 12 [16:55:45:833552]:[Metadata] SceneDatasetAttributesManager.cpp(35)::createObject : File (default) not found, so new default dataset attributes created and registered. [16:55:45:833561]:[Metadata] MetadataMediator.cpp(120)::createSceneDataset : Dataset default successfully created. [16:55:45:833572]:[Metadata] AttributesManagerBase.h(380)::createFromJsonOrDefaultInternal : : Proposing JSON name : ./data/default.physics_config.json from original name : ./data/default.physics_config.json| This file does not exist. [16:55:45:833586]:[Metadata] PhysicsAttributesManager.cpp(26)::createObject : File (./data/default.physics_config.json) not found, so new default physics manager attributes created and registered. [16:55:45:833594]:[Metadata] MetadataMediator.cpp(203)::setActiveSceneDatasetName : Previous active dataset changed to default successfully. [16:55:45:833601]:[Metadata] MetadataMediator.cpp(175)::setCurrPhysicsAttributesHandle : Old physics manager attributes changed to ./data/default.physics_config.json successfully. [16:55:45:833609]:[Metadata] MetadataMediator.cpp(66)::setSimulatorConfiguration : Set new simulator config for scene/stage : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb and dataset : default which is currently active dataset. Renderer: AMD OLAND (DRM 2.50.0, 5.15.0-72-generic, LLVM 12.0.0) by AMD OpenGL version: 4.5 (Core Profile) Mesa 21.2.6 Using optional features: GL_ARB_vertex_array_object GL_ARB_ES2_compatibility GL_ARB_separate_shader_objects GL_ARB_robustness GL_ARB_texture_storage GL_ARB_invalidate_subdata GL_ARB_texture_storage_multisample GL_ARB_multi_bind GL_ARB_direct_state_access GL_ARB_get_texture_sub_image GL_ARB_texture_filter_anisotropic GL_KHR_debug Using driver workarounds: no-layout-qualifiers-on-old-glsl mesa-implementation-color-read-format-dsa-explicit-binding mesa-dsa-createquery-except-pipeline-stats mesa-forward-compatible-line-width-range [16:55:45:889202]:[Metadata] MetadataMediator.cpp(297)::getSceneInstanceAttributesByName : Dataset : default has no preloaded SceneInstanceAttributes or StageAttributes named : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb so loading/creating a new StageAttributes with this name, and then creating a SceneInstanceAttributes with the same name that references this stage. [16:55:45:889222]:[Metadata] AttributesManagerBase.h(380)::createFromJsonOrDefaultInternal : : Proposing JSON name : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.stage_config.json from original name : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb| This file does not exist. [16:55:45:889274]:[Metadata] AbstractObjectAttributesManagerBase.h(184)::createObject : File (data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb) exists but is not a recognized config filename extension, so new default Stage Template attributes created and registered. [16:55:45:889311]:[Metadata] SceneDatasetAttributes.cpp(47)::addNewSceneInstanceToDataset : Dataset : 'default' : Stage Attributes 'data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb' specified in Scene Attributes exists in dataset library. [16:55:45:889320]:[Metadata] SceneDatasetAttributes.cpp(84)::addNewSceneInstanceToDataset : Dataset : 'default' : Lighting Layout Attributes 'no_lights' specified in Scene Attributes but does not exist in dataset, so creating. [16:55:45:889331]:[Metadata] AttributesManagerBase.h(380)::createFromJsonOrDefaultInternal : : Proposing JSON name : no_lights.lighting_config.json from original name : no_lights| This file does not exist. [16:55:45:889345]:[Metadata] LightLayoutAttributesManager.cpp(36)::createObject : File (no_lights) not found, so new default light layout attributes created and registered. [16:55:45:889355]:[Sim] Simulator.cpp(237)::createSceneInstance : Navmesh file location in scene instance : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.navmesh [16:55:45:889368]:[Sim] Simulator.cpp(242)::createSceneInstance : Loading navmesh from data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.navmesh [16:55:45:889384]:[Nav] PathFinder.cpp(852)::loadNavMesh : NavMeshSettings aren't present, guessing that they are the default [16:55:45:889446]:[Sim] Simulator.cpp(244)::createSceneInstance : Navmesh Loaded. [16:55:45:889466]:[Scene] SceneGraph.h(85)::createDrawableGroup : Created DrawableGroup: [16:55:45:889475]:[Assets] ResourceManager.cpp(297)::loadSemanticSceneDescriptor : SceneInstance : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb proposed Semantic Scene Descriptor filename : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.house [16:55:45:914145]:[Assets] ResourceManager.cpp(308)::loadSemanticSceneDescriptor : SSD with SceneInstanceAttributes-provided name data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.house successfully found and loaded [16:55:45:914181]:[Assets] ResourceManager.cpp(1641)::ConfigureImporterManagerGLExtensions : Importing Basis files as ASTC 4x4. [16:55:45:914239]:[Sim] Simulator.cpp(302)::createSceneInstance : Using scene instance-specified Light key : - no_lights - [16:55:45:914255]:[Metadata] MetadataMediator.cpp(66)::setSimulatorConfiguration : Set new simulator config for scene/stage : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb and dataset : default which is currently active dataset. [16:55:45:914276]:[Sim] Simulator.cpp(412)::instanceStageForSceneAttributes : Start to load stage named : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb with render asset : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb and collision asset : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb [16:55:45:914308]:[Assets] ResourceManager.cpp(691)::createStageAssetInfosFromAttributes : Frame :{"up":[0,0,1],"front":[0,1,0],"origin":[0,0,0]} for render mesh named : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb|{"up":[0,0,1],"front":[0,1,0],"origin":[0,0,0]} for semantic mesh named : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy_semantic.ply with type specified as semantic|Semantic Txtrs : False [16:55:45:914321]:[Assets] ResourceManager.cpp(431)::loadStage : Loading Semantic Stage mesh : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy_semantic.ply [16:55:45:914329]:[Scene] SceneGraph.h(85)::createDrawableGroup : Created DrawableGroup: [16:55:45:914337]:[Assets] ResourceManager.cpp(973)::loadStageInternal : Attempting to load stage data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy_semantic.ply [16:55:45:914345]:[Assets] ResourceManager.cpp(852)::loadRenderAsset : Loading Semantic Mesh asset named: data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy_semantic.ply [16:55:45:914352]:[Assets] ResourceManager.cpp(1641)::ConfigureImporterManagerGLExtensions : Importing Basis files as ASTC 4x4. [16:55:49:572259]:[Assets] ResourceManager.cpp(464)::loadStage : Semantic Stage mesh : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy_semantic.ply loaded. [16:55:49:572293]:[Assets] ResourceManager.cpp(507)::loadStage : Start load render asset data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb . [16:55:49:572301]:[Assets] ResourceManager.cpp(973)::loadStageInternal : Attempting to load stage data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb [16:55:49:572315]:[Assets] ResourceManager.cpp(855)::loadRenderAsset : Loading general asset named: data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb [16:55:49:572323]:[Assets] ResourceManager.cpp(1641)::ConfigureImporterManagerGLExtensions : Importing Basis files as ASTC 4x4. [16:55:50:527481]:[Assets] ResourceManager.cpp(2089)::loadMaterials : Building 23 materials for asset named '17DRP5sb8fy.glb' : [16:55:50:527549]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 00:Flat. [16:55:50:527569]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 01:Flat. [16:55:50:527584]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 02:Flat. [16:55:50:527598]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 03:Flat. [16:55:50:527612]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 04:Flat. [16:55:50:527625]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 05:Flat. [16:55:50:527639]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 06:Flat. [16:55:50:527652]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 07:Flat. [16:55:50:527665]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 08:Flat. [16:55:50:527678]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 09:Flat. [16:55:50:527690]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 10:Flat. [16:55:50:527704]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 11:Flat. [16:55:50:527717]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 12:Flat. [16:55:50:527730]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 13:Flat. [16:55:50:527743]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 14:Flat. [16:55:50:527755]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 15:Flat. [16:55:50:527768]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 16:Flat. [16:55:50:527781]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 17:Flat. [16:55:50:527793]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 18:Flat. [16:55:50:527806]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 19:Flat. [16:55:50:527819]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 20:Flat. [16:55:50:527831]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 21:Flat. [16:55:50:527844]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 22:Flat. [16:55:50:566126]:[Sim] Simulator.cpp(439)::instanceStageForSceneAttributes : Successfully loaded stage named : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb [16:55:50:566175]:[Sim] Simulator.cpp(205)::reconfigure : CreateSceneInstance success == true for active scene name : data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb with renderer. [16:55:50:567250]:[Sensor] AudioSensor.cpp(24)::AudioSensorSpec : [Audio] AudioSensorSpec constructor [16:55:50:567286]:[Sensor] AudioSensor.cpp(32)::sanityCheck : [Audio] SanityCheck the audio sensor spec [16:55:50:567296]:[Sensor] AudioSensor.cpp(46)::AudioSensor : [Audio] AudioSensor constructor [16:55:50:567304]:[Sensor] AudioSensor.cpp(32)::sanityCheck : [Audio] SanityCheck the audio sensor spec [16:55:50:567311]:[Sensor] AudioSensor.cpp(48)::AudioSensor : [Audio] Acoustics Configs : sampleRate:16000, frequencyBands:4, directSHOrder:3, indirectSHOrder:1, threadCount:1, updateDt:0.02, irTime:1, unitScale:1, globalVolume:0.25, listenerRadius:0.1, indirectRayCount:500, indirectRayDepth:200, sourceRayCount:200, sourceRayDepth:10, maxDiffractionOrder:10, direct:1, indirect:1, diffraction:1, transmission:1, temporalCoherence:1, meshSimplification:0, dumpWaveFiles:0, enableMaterials:1, writeIrToFile:0 [16:55:50:567331]:[Sensor] AudioSensor.cpp(50)::AudioSensor : [Audio] Channel Layout : channelType:3, channelCount:2 [16:55:50:567338]:[Sensor] AudioSensor.cpp(55)::AudioSensor : [Audio] output directory not provided, will use /home/AudioSimulation [16:55:50:567345]:[Sensor] AudioSensor.cpp(60)::AudioSensor : [Audio] OutputDirectory : /home/AudioSimulation 2023-05-25 16:55:50,567 Initializing task AudioNav Goals [NavigationGoal(position=[-8.308625, 0.19157800000000008, -4.2699], radius=1e-05), NavigationGoal(position=[-8.308625, 0.09157800000000008, -3.2699], radius=1e-05)] [16:55:51:561890]:[Sensor] AudioSensor.cpp(81)::setAudioSourceTransform : [Audio] Setting the audio source position : Vector(-8.30863, 1.69158, -4.2699)] [16:55:51:571219]:[Sensor] AudioSensor.cpp(90)::setAudioListenerTransform : [Audio] Setting the agent transform : position [ Vector(3.00611, 1.57245, -2.67867) ], rotQuat[ Vector(1, 0, 2.45858e-05, 0) ] [16:55:51:571262]:[Sensor] AudioSensor.cpp(278)::createAudioSimulator : [Audio] Create audio simulator iteration: 0 [16:55:51:571535]:[Sensor] AudioSensor.cpp(114)::runSimulation : [Audio] Running the audio simulator [16:55:51:571548]:[Sensor] AudioSensor.cpp(119)::runSimulation : [Audio] New initialization, will upload geometry and add the source at position : Vector(-8.30863, 1.69158, -4.2699) [16:55:51:571559]:[Sensor] AudioSensor.cpp(126)::runSimulation : [Audio] Loading semantic scene [16:55:51:571566]:[Sensor] AudioSensor.cpp(293)::loadSemanticMesh : [Audio] Loading semantic mesh [16:55:52:116342]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 20028 , Material : appliances , LastUpdate : false [16:55:52:116463]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 77148 , Material : shelving , LastUpdate : false [16:55:52:117081]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 1121892 , Material : floor , LastUpdate : false [16:55:52:122754]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 82611 , Material : lighting , LastUpdate : false [16:55:52:122971]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 20019 , Material : blinds , LastUpdate : false [16:55:52:123035]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 27876 , Material : clothes , LastUpdate : false [16:55:52:123276]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 90030 , Material : bed , LastUpdate : false [16:55:52:123556]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 187962 , Material : void , LastUpdate : false [16:55:52:124007]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 24354 , Material : , LastUpdate : false [16:55:52:124391]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 922128 , Material : ceiling , LastUpdate : false [16:55:52:131320]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 61065 , Material : counter , LastUpdate : false [16:55:52:131587]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 287475 , Material : sofa , LastUpdate : false [16:55:52:132367]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 51462 , Material : table , LastUpdate : false [16:55:52:132585]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 152025 , Material : shower , LastUpdate : false [16:55:52:132956]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 27489 , Material : furniture , LastUpdate : false [16:55:52:133308]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 827151 , Material : curtain , LastUpdate : false [16:55:52:135532]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 404010 , Material : chair , LastUpdate : false [16:55:52:145935]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 372825 , Material : misc , LastUpdate : false [16:55:52:146995]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 379536 , Material : door , LastUpdate : false [16:55:52:147909]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 35526 , Material : towel , LastUpdate : false [16:55:52:148027]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 66945 , Material : sink , LastUpdate : false [16:55:52:148408]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 64806 , Material : cabinet , LastUpdate : false [16:55:52:148618]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 130371 , Material : mirror , LastUpdate : false [16:55:52:148996]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 192525 , Material : objects , LastUpdate : false [16:55:52:149454]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 23367 , Material : chest_of_drawers , LastUpdate : false [16:55:52:149541]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 70173 , Material : stool , LastUpdate : false [16:55:52:149886]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 99315 , Material : tv_monitor , LastUpdate : false [16:55:52:150150]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 76977 , Material : toilet , LastUpdate : false [16:55:52:150619]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 119859 , Material : cushion , LastUpdate : false [16:55:52:151993]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 2880879 , Material : wall , LastUpdate : false [16:55:52:179674]:[Sensor] AudioSensor.cpp(398)::loadSemanticMesh : [Audio] Vertex count : 1572308 , Index count : 150918 , Material : picture , LastUpdate : true [16:55:54:911571]:[Sensor] AudioSensor.cpp(143)::runSimulation : [Audio] Adding source at position : Vector(-8.30863, 1.69158, -4.2699) [16:55:54:911636]:[Sensor] AudioSensor.cpp(151)::runSimulation : Running simulation, folder : /home/AudioSimulation0 [16:55:54:916729]:[Sensor] AudioSensor.cpp(259)::getObservationSpace : [Audio] getObservationSpace -> [ChannelCount] : 2 , [SampleCount] : 5602 [16:55:54:920038]:[Sensor] AudioSensor.cpp(90)::setAudioListenerTransform : [Audio] Setting the agent transform : position [ Vector(1.69138, 1.49122, 0.730099) ], rotQuat[ Vector(6.12323e-17, 0, 1, 0) ] [16:55:54:920067]:[Sensor] AudioSensor.cpp(278)::createAudioSimulator : [Audio] Create audio simulator iteration: 1 [16:55:54:920078]:[Sensor] AudioSensor.cpp(114)::runSimulation : [Audio] Running the audio simulator [16:55:54:920086]:[Sensor] AudioSensor.cpp(151)::runSimulation : Running simulation, folder : /home/AudioSimulation1 [16:55:54:922062]:[Sensor] AudioSensor.cpp(259)::getObservationSpace : [Audio] getObservationSpace -> [ChannelCount] : 2 , [SampleCount] : 5591 {'distance_to_goal': 12.327384948730469, 'normalized_distance_to_goal': None, 'success': 0.0, 'spl': 0.0, 'softspl': 0.0, 'top_down_map': None} [16:56:21:623346]:[Sensor] AudioSensor.cpp(90)::setAudioListenerTransform : [Audio] Setting the agent transform : position [ Vector(1.69138, 1.57245, 0.980099) ], rotQuat[ Vector(6.12323e-17, 0, 1, 0) ] [16:56:21:623387]:[Sensor] AudioSensor.cpp(278)::createAudioSimulator : [Audio] Create audio simulator iteration: 2 [16:56:21:623399]:[Sensor] AudioSensor.cpp(114)::runSimulation : [Audio] Running the audio simulator [16:56:21:623407]:[Sensor] AudioSensor.cpp(151)::runSimulation : Running simulation, folder : /home/AudioSimulation2 [16:56:21:625484]:[Sensor] AudioSensor.cpp(259)::getObservationSpace : [Audio] getObservationSpace -> [ChannelCount] : 2 , [SampleCount] : 5596 `` {'distance_to_goal': 12.466228485107422, 'normalized_distance_to_goal': 0.6561172886898643, 'success': 0.0, 'spl': 0.0, 'softspl': 0.0, 'top_down_map': {'map': array([[0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], ..., [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0]], dtype=uint8), 'fog_of_war_mask': array([[0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], ..., [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0]], dtype=uint8), 'agent_map_coord': (384, 801), 'agent_angle': 6.283185307179586}} pointgoal_with_gps_compass: [11.294356 -2.0542433]

kksinghal commented 1 year ago

Also, sorry if the following question is trivial. If we have multiple sound source locations, can I call audio_sensor.setAudioSourceTransform(source_pos + np.array([0, 1.5, 0])) # add 1.5m to the height calculation (taken from examples/soundspaces2_quick_tutorial.ipynb) multiple times (say 2) for different positions to add two sound sources. Thanks

UPDATE: Looks like this doesn't work. setAudioSourceTransform function updates the source location, not add to it.

kksinghal commented 1 year ago

Asnwers are here https://github.com/facebookresearch/sound-spaces/issues/84#issuecomment-1397860379. and https://github.com/facebookresearch/sound-spaces/issues/86#issuecomment-1210060378 Thanks