search

microsoft / LQ-Nets

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
MIT License
239 stars 69 forks source link

Is it possible the BTxB is not invertible? #2

Closed kakaxi314 closed 5 years ago

kakaxi314 commented 6 years ago

https://github.com/Microsoft/LQ-Nets/blob/ca764ae6a5ba9de0627aee2e7fe68c9cf3875305/learned_quantization.py#L92

EowinYe commented 5 years ago

Yes, it is. However, we initialize the basis properly. And it's stable during the training. So the BTxB is always invertible in most cases.

hustzxd commented 5 years ago

I have reimplemented LQ in MxNet, and use Xavier() to initialize the weights of the network. Sometimes the BTxB is a singular matrix, but it's ok when I try again.

monkeyking commented 5 years ago

I have reimplemented LQ in MxNet, and use Xavier() to initialize the weights of the network. Sometimes the BTxB is a singular matrix, but it's ok when I try again.

@hustzxd I tried you implement , the program stuck here.

INFO:root:Namespace(activation_bits=2, batch_size=100, drop_rate=0.0, lr=0.1, lr_decay=0.1, lr_decay_epoch='80,160,300', lr_decay_period=0, mode=None, model='vgg_small_lq', momentum=0.9, num_epochs=400, num_gpus=1, num_workers=4, print_params=False, resume_from=None, save_dir='/home/jackschora/Documents/LQ-Nets-MXNet/saved_model/', save_period=10, save_plot_dir='.', wd=0.0005, weight_bits=1)

GPU usage is 507MB. mxnet-cu100 gluoncv 0.4.0.post0 tensorflow 1.12 gpu 1080TI

hustzxd commented 5 years ago

@monkeyking Thanks for your test. Is there more information about the error? By the way, I just opened the issue of the project. https://github.com/hustzxd/LQ-Nets-MXNet/issues

monkeyking commented 5 years ago

@monkeyking Thanks for your test. Is there more information about the error? By the way, I just opened the issue of the project. https://github.com/hustzxd/LQ-Nets-MXNet/issues

@hustzxd it seemed a bug to VSCODE..... Just can`t debug now..runs well