Closed zeek-han closed 4 years ago
Please check issue-15 . Also for submission, you can convert all the negative values to 0 if that is mandatory or causes any issue. Regarding all negative values, that shouldn't be case. For same user value should be high positives. I can confirm there is no issue with code in that regard.
Thank you for your answer... I think the issue 15 which you mentioned above is different with mine... actually, the score file I wrote is derived from the source code which was updated on 1st of August, while the issue 15 was in May.
I guess that the reason of negative values in the score file is the multiplication of -1 in SpeakerNet.py In SpeakerNet.py, there is the function named "evaluateFromListSave()", and in the function, there is calculating module for distance and score. At that time, score is multiplied by -1. Below is the part of the source code.
dist = F.pairwise_distance(ref_feat.unsqueeze(-1).expand(-1,-1,num_eval), com_feat.unsqueeze(-1).expand(-1,-1,num_eval).transpose(0,2)).detach().cpu().numpy();
score = -1 * numpy.mean(dist);
The scores do not need to be between 0 and 1 for the challenge. This will be corrected.
Thank you very much!!!!
@zeek-han fyi, that code change should not impact values being negative. In fact, you can replace pairwise_distance
with cosine_similarity
and remove score = -1 * numpy.mean(dist)
and you'd still get negative values for unmatched users.
Hello, I think this github page is very closely related to VOXSRC 2020.... In VoxSRC 2020 development toolkit, it requires score file, and on the development toolkit website, " score; must be a floating point number between the closed interval [0, 1], where 1 means the pair of segments correspond to the same speaker and 0 means the pair of segments correspond to different speakers."
but in the code, when I evaluate my model, the written score file has ALL NEGATIVE score, not between [0, 1]. my score file starts with -0.6494 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00008.wav -1.3724 id10270/x6uYqmx31kE/00001.wav id10300/ize_eiCFEg0/00003.wav -0.7400 id10270/x6uYqmx31kE/00001.wav id10270/GWXujl-xAVM/00017.wav -1.2865 id10270/x6uYqmx31kE/00001.wav id10273/0OCW1HUxZyg/00001.wav -0.6512 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00022.wav -1.1112 id10270/x6uYqmx31kE/00001.wav id10284/Uzxv7Axh3Z8/00001.wav -0.6810 id10270/x6uYqmx31kE/00001.wav id10270/GWXujl-xAVM/00033.wav -1.0992 id10270/x6uYqmx31kE/00001.wav id10284/7yx9A0yzLYk/00029.wav -0.7889 id10270/x6uYqmx31kE/00002.wav id10270/5r0dWxy17C8/00026.wav -1.1319 id10270/x6uYqmx31kE/00002.wav id10285/m-uILToQ9ss/00009.wav
Is there any problem??? Is everything OK? with this code to participate in VOXSRC 2020? Thank you.