search

sharathadavanne / seld-dcase2019

Benchmark for sound event localization task of DCASE 2019 challenge
Other
73 stars 29 forks source link

About the format of submission #11

Closed GilgameshD closed 5 years ago

GilgameshD commented 5 years ago

Hi! I have one question that is not related to the code but the challenge submission.

I am a little confused about how to submit systems. How should I provide the results of both datasets (FOA and MIC)? For one submission, the name of the folder should be 'Adavanne_TAUtask3i', where i represents the index of the system. So the results of both dataset should be provided in one submission?

sharathadavanne commented 5 years ago

Hi @GilgameshD each participant is allowed up to four different submissions per task. So ideally, you could submit GilgameshD_CMU_task3_1 for FOA and GilgameshD_CMU_task3_2 for MIC formats. You can accordingly update the metadata files (*.yaml) about your system details, where you mention which format of audio you chose for the corresponding results.

Did this answer your question? :)

GilgameshD commented 5 years ago

Thanks for your reply. So according to my understanding, each submission should provide the result of only one dataset (FOA or MIC). Is that right?

sharathadavanne commented 5 years ago

Yes. The idea is that each of the four submissions will have results of just one method/configuration. So if you have trained your method with just MIC, just FOA, or both MIC and FOA datasets your method will have one single output which you will submit as one submission.

GilgameshD commented 5 years ago

Thanks a lot for your kind reply!

zjysteven commented 5 years ago

@sharathadavanne I'm also experiencing a submission issue. The CMT system says that one could upload 1 to 3 zip files, with maximum file size of 20Mb. However, one system with 500 output files (400 for development and 100 for evaluation) already takes up about 16Mb. Then how am I supposed to submit four systems within one zip file as instructed by the submission page of DCASE2019? Thank you!

sharathadavanne commented 5 years ago

Dont worry @zjysteven :) I have asked the organizers to increase the file upload size restrictions. I will update you here when that happens (Alternatively, drop me an email, I can update you there aswell). Worst case we will figure out some way to get your results to us :)

zjysteven commented 5 years ago

@sharathadavanne Oh, great! Thanks for your help!

sharathadavanne commented 5 years ago

Hi @zjysteven try submitting now. The maximum file size has been increased to 100 Mb.

zjysteven commented 5 years ago

@sharathadavanne Have successfully submitted! Thanks again!