search

facebookresearch / vicreg

VICReg official code base
MIT License
523 stars 88 forks source link

Questions about loss's range of experience #14

Closed benyang0506 closed 2 years ago

benyang0506 commented 2 years ago

In my experiment, the cov_loss is about 1e-5 within the first 10 epochs and then became 0, I wonder the magnitude of your cov_loss. Thank you!

Adrien987k commented 2 years ago

Hi,

A loss of 0 most likely indicates a collapse of the representations. the magnitude of the loss depends on your data, but the cov_loss should be stable throughout the training, only the inv_loss significantly decreases.

dc250601 commented 1 year ago

In my case the Cov_loss is slowly increasing while the other two are slowly going down. @Adrien987k Is it normal to see such a phenomenon? In about 20 epochs it has increased from 2.6 to 6.5.