search

Eric-mingjie / rethinking-network-pruning

Rethinking the Value of Network Pruning (Pytorch) (ICLR 2019)
MIT License
1.51k stars 293 forks source link

Network slimming loss function #49

Closed ivllm closed 3 years ago

ivllm commented 3 years ago

Hi, Thank you for sharing a good experiment. I have a question about the loss function of network slimming.

The paper shows the training objective as shown below. image

But codes only use cross entropy function when training after pruning. Is this the right implementation? Please explain if I misunderstood.

Eric-mingjie commented 3 years ago

The network slimming loss is incorporated in the code, see this function https://github.com/Eric-mingjie/rethinking-network-pruning/blob/master/cifar/network-slimming/main.py#L122. We update the scaling factor explicitly.

ivllm commented 3 years ago

I understand. Thank you very much for your quick reply!