search

buaabai / Ternary-Weights-Network

Ternay-Weights-Network/Pytorch
36 stars 12 forks source link

Introduce Ternary Weight Networks (TWNs) - neural networks with weights constrained to +1, 0 and -1. #2

Open manwu1994 opened 3 years ago

manwu1994 commented 3 years ago

Hi. I try to get the weight (-1,0,1) based on your code and the TWNs paper, I can't get the (-1,0,1) weight, but can get the (-a,0,a). Could you give me any suggestions? Thank you so much for your time.

buaabai commented 3 years ago

a应该是论文中的scaling factor，也就是代码中的alpha，每层权重应该对应着同一个alpha，将每层的alpha提取出来就可以得到（-1，0，1）这样的权重了。 a is supposed to be the scaling factor in the paper, which is corresponded by alpha in codes. each layer has its own alpha, u can divide weights by alpha to get weights like (-1, 0, 1).

manwu1994 commented 3 years ago

感谢回复！明白了！但是好像不是每一层对应同一个alpha，而是每一个神经元对应一个alpha，因此想问一下您有试过提取alpha参数吗？非常感谢！

manwu1994 commented 3 years ago

感谢回复！明白了！但是好像不是每一层对应同一个alpha，而是每一个神经元对应一个alpha，因此想问一下您有试过提取alpha参数吗？非常感谢！

嗯嗯不好意思我理解有误，是一层对应一个alpha,那如果要获取(-1,0,1)的权重，还得提取delta的参数，对吧？谢谢您的时间！