search

taoyang1122 / pytorch-SimSiam

A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.
Apache License 2.0
78 stars 8 forks source link

network initialization #5

Closed ruotianluo closed 3 years ago

ruotianluo commented 3 years ago

Hi,

I am wondering why you added the reset_parameters.https://github.com/taoyang1122/pytorch-SimSiam/blob/main/models/simsiam.py#L83

taoyang1122 commented 3 years ago

In the supplementary the author said they do an uniform initialization for FC and Conv layers.

ruotianluo commented 3 years ago

I see, didn't notice that, sorry. Thank you so much.

ruotianluo commented 3 years ago

Wait, I just checked the paper. The paper says they are using default pytorch initializer, so you don't need to write a customized reset_parameter. but you do need resnet50(zero_init_residual=True) It may affect final result.

taoyang1122 commented 3 years ago

Yes, thanks for pointing that out.