search

TreB1eN / InsightFace_Pytorch

Pytorch0.4.1 codes for InsightFace
MIT License
1.73k stars 423 forks source link

Do huge batch size necessary? #27

Open Jasonbaby opened 5 years ago

Jasonbaby commented 5 years ago

Hi, I have tried to retrained the model with batch size 50, But it seems the model do not converge. I am wondering if the batch size is needed to be huge (more than 100)?

TreB1eN commented 5 years ago

I don't think so, maybe you need to tune your lr if you change the batchsize

TreB1eN commented 5 years ago

and about convergence, it will not converge at all after several epoches

Jasonbaby commented 5 years ago

Thanks a lot. Actually, I have trained 3 epoches, with emore dataset. And the loss does not decrease at all. Is it ok? So I continue my training? I had changed some codes, but not changed the Arcface head.

Jasonbaby commented 5 years ago

I changed the backbone model, and the re-train the model with emore dataset.

TreB1eN commented 5 years ago

that's not OK, the loss should start to decrease at very early stage

Jasonbaby commented 5 years ago

Thanks a lot. I will recheck my code.

yxchng commented 5 years ago

@Jasonbaby Have u solved the problem? I am also training with small batch size 64 with r34 backbone and it is not converging.

joe-zxh commented 2 years ago

I think the batch size need to be large, since there are a lot of batch normalization in mobileFaceNet.