search

Pokemon-Huang / sparse-structure-selection-PyTorch

PyTorch-Implementation of "Data-Driven Sparse Structure Selection for Deep Neural Networks"
20 stars 2 forks source link

After training, the parameter does not become 0, what should i do? #1

Open panchengl opened 3 years ago

panchengl commented 3 years ago

hello, thanks your work, i use ypur demo to train, but after training, the parameter does not become zero, image

panchengl commented 3 years ago

lambda 'Parameter containing: tensor([0.6660, 0.4774, 0.5917, 0.8245, 0.9268, 2.6580], device='cuda:0', requires_grad=True)'

panchengl commented 3 years ago

@Pokemon-Huang Looking forward to your reply， thank you

Pokemon-Huang commented 3 years ago

@panchengl Are u running the algorithm on ResNet-20? I did remember this work can not prune this model, may be it's too shallow, try ResNet50.

panchengl commented 3 years ago

thanks your reply, yes, i use resnet-20, may i use resnet56 in ciafar10 or cifar 100? or resnet50 in imagenet?

Pokemon-Huang commented 3 years ago

resnet56 in cifar10 is ok, i didn't try the other either. Generally speaking, the algorithm works better with easier datasets and deeper models.

panchengl commented 3 years ago

ok, thank u , i want use blocks prune in object detections, Do you think this algorithm is effective?

Pokemon-Huang commented 3 years ago

emmm, the unique optimization method of this algorithm will influence the accuracy, so it will work if your datasets is easy. BTW, this work publics in 2017, u can check some recent papers, like nas.

panchengl commented 3 years ago

Ok, thanks for your advices, i will try in my datasets, if work, i will reply to you ,  thank u

---Original--- From: "Pokemon-Huang"<notifications@github.com> Date: Sun, Dec 27, 2020 18:20 PM To: "Pokemon-Huang/sparse-structure-selection-PyTorch"<sparse-structure-selection-PyTorch@noreply.github.com>; Cc: "panchengl"<2943499076@qq.com>;"Mention"<mention@noreply.github.com>; Subject: Re: [Pokemon-Huang/sparse-structure-selection-PyTorch] After training, the parameter does not become 0, what should i do? (#1)

emmm, the unique optimization method of this algorithm will influence the accuracy, so it will work if your datasets is easy. BTW, this work publics in 2017, u can check some recent papers, like nas.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

Ta-SIR commented 3 years ago

hello, thanks your work, why is there no R1 loss in main，but in sss