search

facebookresearch / sound-spaces

A first-of-its-kind acoustic simulation platform for audio-visual embodied AI research. It supports training and evaluating multiple tasks and applications.
https://soundspaces.org
Creative Commons Attribution 4.0 International
338 stars 55 forks source link

what's the rule of having IR data in a scene? #66

Open sukun1045 opened 2 years ago

sukun1045 commented 2 years ago

Hi, I am trying to see which coordinate points in a scene may have IR files. For example, I am looking at the room_0 scene in Replica. The top-down map looks like this: room_0_topdown

And according to the coordinates given by the metadata and the IR file names. I got something like this: room_0_IR where the blue dots are coordinates of all points in mesh coordinates and red dots are the points where have IR.

It seems like the points available for IR are less than all points. Do I miss something when I download the data? If not, according to the first figure, the white areas are regions where agents cannot reach, like tables, chairs, etc. I am wondering why there isn't any IR on those points? While it is not reachable for agent, the sound should still be able to propagate through these objects, right? I am wondering what's the rule for generating IR in a scene.

Looking forward to your reply. Thank you.

ChanganVR commented 2 years ago

@sukun1045 sorry for the late reply. I totally missed your post until today. The IRs uploaded have been pruned to reduce disk space and thus they only contain points that are identified as reachable by other points. So I have removed points on furnitures as well as isolated points. The pruning strategy is described in the paper and also this file contains the script for pruning the matterport3d dataset: https://github.com/facebookresearch/sound-spaces/blob/main/scripts/generate_matterport3d_metadata.py.

sukun1045 commented 2 years ago

@ChanganVR thanks for your reply. I saw the details in the paper appendix. Just want to make sure I understand it correctly. To do the acoustic simulation, you first do the grid construction and prune grid points in inaccessible locations. Then with the remaining grid points, you perform the bidirectional path tracing algorithm to do the acoustic simulation. Is that correct?

ChanganVR commented 2 years ago

@sukun1045 we first did grid construction and acoustic simulation for all pairs of grid points in the space. Since some points are not navigable, we pruned these points for the navigation task.

The order here actually does not really matter. But let me know if you have other questions.

sukun1045 commented 2 years ago

@ChanganVR Thanks for your reply. It is clear now. Final question: is the point.txt in the metadata describing all gird points you consider for acoustic simulation?

ChanganVR commented 2 years ago

Yes. That is true. And graphs contain the pruned points.