Inquiry: Pruning process without spatial decomposition?

[slothkong]

Thanks again for sharing your code and releasing the VGG-16 5X pruned caffemodel. May I ask how much should the code be changed in order to produce this model (pruning only, no 3C included) ?

Can this be done by simply passing a different flag instead of "-action c3"? Thanks!!!! And sorry for asking but I'm sure a lot of people will want to know about this.

[ethanhe42]

The code was rewrite once, I need to adapt it to current released code. actually you can implement it yourself, just delete VH and ITQ in R3 function. Duplicate of #16

[slothkong]

Thanks!!!

[OPPOA113]

@slothkong Hi Have you successfully changed the code for pruning only?And what should be modify.

[slothkong]

@OPPOA113 I'm working on it right now. I also want to modify the code to support other DNNs . I think what @yihui-he was talking about can be found in https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1343 and https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1376 . Changing those two "if-conditions" to "False" might be the solution. Ahh, and also https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1373 should be deleted, since now we don't insert any layers into the model. Ill post if I get it working.

[slothkong]

@OPPOA113 , seems like "channel pruning" depends on "channel decomposition" ㅜㅜ. There is a set of additional layer names conv_H, conv_V and conv_P https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1334-L1336

We are not interested in conv_H or conv_V, only conv_P. However, conv_P is introduced in the new model self.WPQ when we execute "channel decomposition" ( calling the function ITQ_decompose() yields the following values: W1, W2, B, W12)

https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1388-L1392

Finally, in "channel pruning", we use conv_P to set self.WPQ

https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1414-L1419

TODO: we have find a way to define W2 without executing ITQ_decompose() . @yihui-he, if you are reading this, please comment^^

--- EDIT: Or maybe we can simply add: P_shape = [W2.shape[0], W2.shape[1], 1, 1] self.WPQ[(conv_P, 0)] = W2.reshape(P_shape) self.WPQ[(conv_P, 1)] = B2

under the code line : https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1407

[OPPOA113]

@slothkong thanks,how it was going on ,it work finally? actually, i do not understand the code completely. if you have finish,please let me know. thank you !! i will follow you work!

[slothkong]

@OPPOA113 still not working, I did this: idxs, W2, B2 = self.dictionary_kernel(X_name, None, d_c, convnext, None) P_shape = [W2.shape[0], W2.shape[1], 1, 1] self.WPQ[(conv_P, 0)] = W2.reshape(P_shape) But I get an error because W2 is too big to be reshaped. Im not sure what should I change.

[slothkong]

@OPPOA113 Nevermind, still trying to figure it out

[zh0ngtian]

@slothkong Sorry to bother you, would you like to tell me whether you implement it ?

[slothkong]

@YaoZhongtian I was unable to debug it. Seems like I never fully understood how @yihui-he's code worked. So even thought I managed to prune the hold model and write the protobuf file, it's accuracy was awefully bad.

The author told he was going to release some code for this after he was done with preparations for some conference. But I guess this repo is dead now

[zh0ngtian]

@slothkong Thank for your reply. I just started trying to solve it.

[Tveek]

@YaoZhongtian any solution for this?

[zh0ngtian]

@Tveek Actually, I can't understand the code completely. Just waiting for the author to release the code of this part.

[bbjy]

Is there anyone know how the speed-up is determined? And what is the relationship between the speed-up ratio and the corresponding channel preserved ratio?Thank you!