search

AojunZhou / Incremental-Network-Quantization

Caffe Implementation for Incremental network quantization
Other
191 stars 74 forks source link

Why does the quantized file size remain unchanged？ #36

Open fantexibaba opened 5 years ago

fantexibaba commented 5 years ago

I've tried 2bit and 5bit quantizations,but the file size is the same as before quantization.

AojunZhou commented 5 years ago

@fantexibaba we don't save the model with low-precision in training process.

fantexibaba commented 5 years ago

@AojunZhou Thank you for your reply.But I still don't get it.I mean,the caffemodel, like 'alexnet_part2_iter_63000.caffemodel',which memory size of the file is about 244MB.After this quantification,I got 'alexnet_part4_iter_1.caffemodel',which is still 244MB.Shouldn't file size be smaller?

vassiliad commented 5 years ago

@fantexibaba think of it as Caffee using specially crafted floating point numbers instead of actual 2-bit (or 5-bit) numbers. Of course, you could do a post-processing step on the Weight blobs and convert them to the quantized format but then you'd need specialized operations which could use this brand new quantized numbering format. Incidentally, this is why the authors/developers of INQ decided to use floating point operations instead of real quantized operations in the first place, otherwise they'd have to implement AND optimize a large subset of Caffee for this quantized numbering format.

fantexibaba commented 5 years ago

@vassiliad Thank you for your reply. It helped me a lot.

vassiliad commented 5 years ago

I'm happy to hear that @fantexibaba

If you want to learn more about the technical aspect of the INQ methodology, take a look at what I wrote over a few months ago in another issue: #28

Shortly after that time I stopped playing around with quantization in Deep-learning so I'm afraid that I won't be able to provide any more information on the matter.

Take care friend!

fantexibaba commented 5 years ago

@AojunZhou Sorry to bother you again,I've done the quantification and it's really useful.But the accuracy I trained under your project is aleways 3% lower than your paper's result.Could you give me some advice to make it better?

AojunZhou commented 5 years ago

@fantexibaba can you share your training setting?

fantexibaba commented 5 years ago

@fantexibaba can you share your training setting? Sure.I've tried to set the base_lr to 0.00001,and the same result was achieved. net: "./examples/INQ/alexnet/train_val.prototxt" test_iter: 1000 test_interval: 3000 base_lr: 0.003 lr_policy: "step" gamma: 0.2 stepsize: 15000 display: 100 max_iter: 63000 momentum: 0.9 weight_decay: 0.0005 snapshot: 3000 snapshot_prefix: "models/bvlc_alexnet/alexnet_part1" solver_mode: GPU