search

sharathadavanne / seld-dcase2022

Baseline method for sound event localization task of DCASE 2022 challenge
50 stars 21 forks source link

Difficulty reproducing results #1

Closed crixus5678 closed 2 years ago

crixus5678 commented 2 years ago

Hi,

I'm wondering if anyone tried to reproduce the results? I tried to run setting 3 and the F1-score was at 0.14 (average of 5 trials). I'm wondering is it because I trained it without the synthetic set (DCASE 2022 simulated data for baseline training - https://doi.org/10.5281/zenodo.6406873) that's why that is quite a big gap with the reported accuracy

Best Regards,

sharathadavanne commented 2 years ago

Hey @crixus5678 you are right, the results we have reported as baseline scores were obtained by training on a mixture of synthetic and real-life data, and testing it on real-life data only.

crixus5678 commented 2 years ago

@sharathadavanne thanks for the fast reply. Let me try it again

Update: Tried again with additional synthetic data and able to get comparable results (ER/F/LE/LR/SELD: 0.65/0.30/22.62/0.48/0.50)