search

albanie / collaborative-experts

Video embeddings for retrieval with natural language queries
https://www.robots.ox.ac.uk/~vgg/research/collaborative-experts/
Apache License 2.0
332 stars 55 forks source link

Random Retrieval Scores #20

Open ra1ph2 opened 3 years ago

ra1ph2 commented 3 years ago

Hey, I have downloaded the YouCook2 features and am training a common space learning network to associate positive sentence-clip pairs for video retrieval. I have loaded the video features and the text features from the pickle files using a custom dataset and then am loading the files using the dictionary provided. I have checked my code for errors and am pretty sure that it is correct since when I use the same training or val set as the training set and then check retrieval scores on it, the model is converging to a good score. But when I validate the model on an unseen set the results are totally random (Eg- MedianR being 1500/3000). So, I just wanted to know if there is some trivial step that I might missing while loading the dataset or if the sentence-pairs are incorrectly matched in the pickle file. Thanks in advance and I hope I am clear in explaining the issue.

RSteendam commented 2 years ago

I'm also curious to this. @ra1ph2 Did you find out what could be causing this?