search

Pokemon-Huang / sparse-structure-selection-PyTorch

PyTorch-Implementation of "Data-Driven Sparse Structure Selection for Deep Neural Networks"
20 stars 2 forks source link

hello, thanks your work, why is there no R1 loss in main，but in sss #2

Open Ta-SIR opened 3 years ago

Pokemon-Huang commented 2 years ago

That is the point, r1 loss is the case for sparsity. This work first train without r1 loss in main, then add it in sss. If we train with r1 loss in the beginning, probably we can not get a better accuracy. Its just like warm up in other algorithm.