search

666DZY666 / micronet

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
MIT License
2.2k stars 478 forks source link

BnFoldConv question #44

Open lovepan1 opened 3 years ago

lovepan1 commented 3 years ago

你好: in quantization/WqAq/IAO/models/util_wqaq.py if self.bias is not None:
bias = reshape_to_bias(self.beta + (self.bias - batch_mean) (self.gamma / torch.sqrt(batch_var + self.eps))) else: bias = reshape_to_bias(self.beta - batch_mean (self.gamma / torch.sqrt(batch_var + self.eps))) weight = self.weight reshape_to_weight(self.gamma / torch.sqrt(self.running_var + self.eps))
为什么torch.sqrt()里面batch_var不使用平方，类似于这种: bias = reshape_to_bias(self.beta + (self.bias - batch_mean) (self.gamma / torch.sqrt(batch_varbatch_var + self.eps))) weight = self.weight reshape_to_weight(self.gamma / torch.sqrt(self.running_var*self.runing_var + self.eps)) 期待您的回复。

ghost commented 3 years ago

您好，您这块是怎么理解的呢，能不能交流下呢

zhangzan1997 commented 3 years ago

这已经计算成方差了，不用平方

666DZY666 commented 3 years ago

batch_var = torch.var(output, dim=dims) 是的，已经是方差了。