search

cvqluu / Angular-Penalty-Softmax-Losses-Pytorch

Angular penalty loss functions in Pytorch (ArcFace, SphereFace, Additive Margin, CosFace)
MIT License
481 stars 92 forks source link

Question about loss_functions.py #10

Open junikkoma opened 3 years ago

junikkoma commented 3 years ago

Hi, thank you for great implementation. I appreciate your work as well as your generosity for opening it.

As mentioned in title, I have a question about line 35 of loss_functions.py, as given below

https://github.com/cvqluu/Angular-Penalty-Softmax-Losses-Pytorch/blob/c41d599622b6a6ba7e5be6faf3a01da1202024ea/loss_functions.py#L35

To my understanding, I think it would initialize new fully connected layer in each epoch of training. I don't understand how this layer can be optimized via backpropagation, as it would be re-initialized each time.

It would be a great help if anyone can teach me why such inference is wrong.

zhekang commented 3 years ago

Hi @junikkoma , nn.Linear is used to initialize the weights of fc layer, and during model training, forward() function is called. The layer only get initialized once here: https://github.com/cvqluu/Angular-Penalty-Softmax-Losses-Pytorch/blob/c41d599622b6a6ba7e5be6faf3a01da1202024ea/models.py#L24