search

RElbers / info-nce-pytorch

PyTorch implementation of the InfoNCE loss for self-supervised learning.
MIT License
445 stars 37 forks source link

multiple positive sampels per query #7

Closed YiJun-Yang closed 1 year ago

YiJun-Yang commented 2 years ago

Hi, Thanks very much for your work. I wonder how should I modify your loss to adapting to the case of multiple positive samples per query. For example, query.shape = (1,128), positive_keys.shape = (5, 128), negative_keys.shape = (5, 128).

Hope your reply

dipta007 commented 2 years ago

You can look into this paper: Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering They have used multiple positives, in short, what they have done is to call infoNCE for each positive with the same negative samples and got the average of them.

Yuntian9708 commented 1 year ago

@YiJun-Yang I just got similar questions. I post my implementations and understandings in the open issue. If you still have a question about it, you can discuss it with me in the open issue!

rainhu22 commented 1 year ago

@YiJun-Yang I just got similar questions. I post my implementations and understandings in the open issue. If you still have a question about it, you can discuss it with me in the open issue!

Have you achieved multiple positive sampels per query？

Yuntian9708 commented 1 year ago

@YiJun-Yang I just got similar questions. I post my implementations and understandings in the open issue. If you still have a question about it, you can discuss it with me in the open issue!

Have you achieved multiple positive sampels per query？

my approach is to let one query calculate similarities with each positive sample and then calculate the average value. I think the meaning of this is to make the query similar to the entire positive sample set.