search

sthalles / PyTorch-BYOL

PyTorch implementation of Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
480 stars 72 forks source link

Representation Collapse #3

Open arjung128 opened 4 years ago

arjung128 commented 4 years ago

First of all, thank you so much for making this work open source!

My question isn't directly related to your implementation but rather a question about the paper -- I didn't know where else to ask, and I figured you'd have a pretty good understanding of the paper. Having said that, feel free to mark it closed if you think it is inappropriate.

I recently read the paper and I don't understand why the network doesn't cheat by simply learning to output 0s, or in their own words, by learning "collapsed representations." Any ideas?

hafnerfe commented 3 years ago

@arjung128 hi, this answer is late, but maybe you'll find this video useful. Hope it helps.