search

xinliu20 / MEC

MIT License
40 stars 2 forks source link

BYOL + BarlowTwins #5

Open mongeoroo opened 3 months ago

mongeoroo commented 3 months ago

Thanks for your great work.

In paper, you said that MEC equals to merging simsiam and BT.

But in the git code, it used momentum encoder, much more similar to merging BYOL and BT.

Do I get it correctly?

xinliu20 commented 3 months ago

Thanks for your interest in our work.

Yes. From the perspective of objective function, BYOL uses mse between normalized vectors, which equals to cosine similarity used in SimSiam. From the perspective of network architecture, we used momentum encoder following BYOL for improved performance, but MEC can also work without momentum encoder, which then equals to SimSiam. So in general, the objective function of MEC can unify batch-wise objectives (including both BYOL and SimSiam) and feature-wise objectives (e.g., BarlowTwins).

-----Original Messages----- From:mongeoroo @.> Send time:Thursday, 05/30/2024 16:23:14 To: xinliu20/MEC @.> Cc: Subscribed @.***> Subject: [xinliu20/MEC] BYOL + BarlowTwins (Issue #5)

Thanks for your great work.

In paper, you said that MEC equals to merging simsiam and BT.

But in the git code, it used momentum encoder, much more similar to merging BYOL and BT.

Do I get it correctly?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

mongeoroo commented 1 month ago

Thank you for your reply!

Could you share the code for transferring to the cifar10 in Table 8?

I want to reproduce your work but the results are undesirable.

Thanks.