search

666DZY666 / micronet

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
MIT License
2.2k stars 478 forks source link

Dorefa训练的反向传播 #96

Open csorujian opened 2 years ago

csorujian commented 2 years ago

请问一下作者，原文中的对gradient的量化是不是没有应用在code中？

666DZY666 commented 2 years ago

是的。梯度量化理论上可以加速，但这里就算做了也还是浮点在反传，不会加速，就没做。 这个梯度量化原文本意是，为在终端训练提供可能。