search

facebookresearch / swav

PyTorch implementation of SwAV https//arxiv.org/abs/2006.09882
Other
1.99k stars 280 forks source link

A question about batch_size and nmb prototypes #118

Open mrFocusXin opened 1 year ago

mrFocusXin commented 1 year ago

I encountered a problem in the process of implementing swav on a small dataset. My batch size was only 16 maximum, but the number of my classes was 150. As your suggestion, the number of nmb prototypes was set to be one order of magnitude more than the number of classes. So my nmb prototypes are 1500. Without queues for the first 15 epochs, each batch contains 15 samples, which is not at all enough to distribute evenly among the 1,500 prototypes. So it's inevitable that there will be empty prototypes, is that normal? Should I ignore the empty prototypes and continue training?

mrFocusXin commented 1 year ago

@mathildecaron31 Sorry to bother you, but have you ever encountered this problem? For example, if the batch size is much smaller than nmb prototypes, how to deal with empty prototypes?