search

smallflyingpig / universal_adversarial_perturbation_generative_network_for_speaker_recognition

code for paper "Universal Adversarial Perturbations Generative Network for Speaker Recognition"
https://smallflyingpig.github.io/UAPs_for_speaker_recognition/main
MIT License
22 stars 7 forks source link

Initialization last convolution layer as zeros #4

Open jiatongli1997 opened 3 years ago

jiatongli1997 commented 3 years ago

I have a problem about your project. If you initialize the last layer as zeros, the gradients will also be zeros for all layers. Then the model cannot be trained. Do i understand correctly?

smallflyingpig commented 3 years ago

I have a problem about your project. If you initialize the last layer as zeros, the gradients will also be zeros for all layers. Then the model cannot be trained. Do i understand correctly?

Hi, I understand what you mean. When the last layer is set as zeros, the gradients before the last layer will be zero. However, the gradient for the last layer is not zero, so the last layer will be optimized firstly. Other layers will be optimized later.