search

facebookresearch / sound-spaces

A first-of-its-kind acoustic simulation platform for audio-visual embodied AI research. It supports training and evaluating multiple tasks and applications.
https://soundspaces.org
Creative Commons Attribution 4.0 International
345 stars 55 forks source link

heard.pth vs unheard.pth #96

Closed sreeharshaparuchur1 closed 1 year ago

sreeharshaparuchur1 commented 1 year ago

Hey,

As mentioned in your readme files, one can evaluate the pretrained av_wan code using either heard.pth or unheard.pth. Could you elucidate upon what exactly the difference was in the creation of these two models and what performance disparity we can expect between models using the given weights?

Thank you.

ChanganVR commented 1 year ago

Hi @sreeharshaparuchur1, these two checkpoints are provided for two different experiments. Heard sound experiment : training and testing on the same telephone ringing sound and unheard sound experiment: training on multiple sounds and testing on unheard multiple sounds for evaluating generalization. Please check table 2 in the AV-WaN paper for more details.

sreeharshaparuchur1 commented 1 year ago

Thanks for the clarification :) the paper did mention that there was a difference between the two checkpoints but this elucidates upon how and why there would be a difference in using either of the checkpoints in evaluating the model.