search

sovrasov / flops-counter.pytorch

Flops counter for convolutional networks in pytorch framework
MIT License
2.82k stars 307 forks source link

Formula of batchnorm #99

Closed unipdjahanifar closed 1 year ago

unipdjahanifar commented 1 year ago

Hello Could you please let me know the formula that is behind the calculation of number of macs for batch normalization?

sovrasov commented 1 year ago

Hi. See https://github.com/sovrasov/flops-counter.pytorch/blob/master/ptflops/pytorch_ops.py#L44

unipdjahanifar commented 1 year ago

Thanks a lot. based on the shred codes, could you please let me know whether this statement is correct: "in batch normalization layer the output is the product of input array element (np.prod) to compute the number of macc?"

sovrasov commented 1 year ago

Basically yes, at inference stage the transform is (x - m) / s

unipdjahanifar commented 1 year ago

Thanks a lot. Could you please let me know whether is there any formula for the backward? your kind support in this regard will be appreciated.

sovrasov commented 1 year ago

All the formulas related to BN can be fount in the original paper: https://arxiv.org/pdf/1502.03167.pdf