Open nikheelpandey opened 3 years ago
def D(p, z, version='simplified'): # negative cosine similarity if version == 'original': z = z.detach() # stop gradient p = F.normalize(p, dim=1) # l2-normalize z = F.normalize(z, dim=1) # l2-normalize return -(p*z).sum(dim=1).mean()
elif version == 'simplified':# same thing, much faster. Scroll down, speed test in __main__
return - F.cosine_similarity(p, z.detach(), dim=-1).mean()
else:
raise Exception
There is a 'detach' after 'z' when compute loss
Hello,
I was using your implementation of SimSiam for contrastive learning. I noticed that the model that you have created has a few problems:
Could you please clarify how and where you are taking care of it?