Closed kai-xie closed 7 years ago
Hi @kai-xie , what about using a smaller c_rate (e.g., 2 and 3) for the 'ip1' layer? In the cases when other layers are dense and only 'ip1' is to be compressed, 4 might be too large.
Yes, the obtained caffemodel after running this repo should be larger than the original dense model. This is because we store both the weight tensor Ws and the mask tensor Ts. Hence you should post-process the obtained model to further get sparse tensor W.*Ts and use sparse tensor storage formats for getting memory/storage savings.
I changed the c_rate to 2 and it worked. As for getting the compressed model, I think I'd better read the source code.
Thank you very much for your help!
DNS is a good method and thank you for sharing your code!
My question is: The compilation and installation was successful and it did not cost much effort ( Only your code was used and the original Caffe code was not added, so I assume your code could be used as a standalone package ). But when I tried to compress the Lenet5, as suggested int the README, I only changed the "ip1" layer's type to "CInnerProduct" and added the "cinner_product_param" part, the result did not converge, and the size of the ouput caffemodel is 3.2M, even larger thant the original size 1.7M.
So I was wondering if you have encountered this kind of problem before and what is possibly my mis-operation.
The following is the prototxt file, as is in the caffe examples, only the "CInnerProduct" part changed:
The output from iteration 9000 to iteration 10000 is as following: (accuracy lingering around 0.1135 )
Thank you very much!