Open zwhus opened 2 months ago
Thanks for your work! I have a question about gradient accumulate on contrastive learning. you use the code as follows:
all_teacher_image_features = torch.cat( [teacher_image_features] + gathered_teacher_image_features[:rank] + gathered_teacher_image_features[rank + 1 :] )
why not use the code
gathered_teacher_image_features[self.rank] = teacher_image_features all_teacher_image_features = torch.cat(gathered_teacher_image_features, dim=0)
Thanks for your work! I have a question about gradient accumulate on contrastive learning. you use the code as follows:
why not use the code