search

joonson / voxconverse

Spot the conversation: speaker diarisation in the wild
119 stars 14 forks source link

labels for the test set #7

Open zhiyunfan opened 3 years ago

zhiyunfan commented 3 years ago

The labels for the test set will be released to public after the VoxCeleb Speaker Recognition Challenge in October 2020. How can we download the labels for the test set? Looking forward to your reply.

scalfs commented 3 years ago

I'm interested in the labels for the test set as well

JaesungHuh commented 3 years ago

The test set labels are now released.

hbredin commented 3 years ago

Thanks @JaesungHuh for sharing the test set labels.

For comparison with official results reported here, can you please confirm that these were computed on the subset of 232 files for which labels are available and not on the whole set of 312 files shared initially? This is important for the speaker diarization community to make sure we are not comparing apples and oranges.

cc @fnlandini @desh2608

JaesungHuh commented 3 years ago

@hbredin Thanks for the question. Yes, the released Voxconverse test set are subset of 232 files from the whole set of 312 files shared initially. We did another few rounds of check to make labels more accurate and removed some files which annotators couldn't be 100% sure of their annotation. Please use this version from now on.

hbredin commented 3 years ago

Thanks for clarifying.

What should we call this version in publications: VoxConverse 2021 ? VoxConverse v0.0.2?

JaesungHuh commented 3 years ago

I'll re-open this issue for other people to see. I have to discuss co-authors about this, but I think either is fine. Will let you know if the term fixed.

JaesungHuh commented 2 years ago

We've recently released ver 0.3, fixing some of the errors in the test set labels. Please call "VoxConverse 0.3" when you use this dataset.

hbredin commented 2 years ago

Thanks for the heads-up @JaesungHuh.

Switching reference labels from 0.2 to 0.3 did "improve" my baseline by a whooping 2.8% (relative) in terms of speaker confusion rate. That is not negligible.

JaesungHuh commented 2 years ago

Yes. We found these errors during the preparation for this year's VoxSRC workshop. I'll re-open this issue to let everyone know about this. I apologize for any inconvenience.

ahmadikalkhorani commented 10 months ago

Where can I find the link to the video files?

folalafish commented 1 week ago

Where should I download the video file that corresponds to the audio file?