integer overflow, when calculate the MACs of the ViT on Windows

[lgylylyct]

when I used ptflops to calculate the MACs of the ViT on Windows computer, I found the MACs of some module is negative.

(2): Block(
  7.09 M, 7.550% Params, -1054860544.0 Mac, -9.472% MACs,
  (norm1): LayerNorm(1.54 k, 0.002% Params, 3.15 MMac, 0.028% MACs, (768,), eps=1e-06, elementwise_affine=True)
  (attn): Attention(
    2.36 M, 2.516% Params, -3221222400.0 Mac, -28.924% MACs,
    (qkv): Linear(1.77 M, 1.887% Params, -1342174976.0 Mac, -12.052% MACs, in_features=768, out_features=2304, bias=True)
    (proj): Linear(590.59 k, 0.629% Params, -1879047424.0 Mac, -16.873% MACs, in_features=768, out_features=768, bias=True)
  )
  (norm2): LayerNorm(1.54 k, 0.002% Params, 3.15 MMac, 0.028% MACs, (768,), eps=1e-06, elementwise_affine=True)
  (mlp): MLPBlock(
    4.72 M, 5.031% Params, 2.16 GMac, 19.396% MACs,
    (lin1): Linear(2.36 M, 2.516% Params, 1.07 GMac, 9.641% MACs, in_features=768, out_features=3072, bias=True)
    (lin2): Linear(2.36 M, 2.514% Params, 1.07 GMac, 9.641% MACs, in_features=3072, out_features=768, bias=True)
    (act): GELU(0, 0.000% Params, 12.58 MMac, 0.113% MACs, approximate='none')
  )
)

I found this bug in this code:

def linear_flops_counter_hook(module, input, output):
    input = input[0]
    # pytorch checks dimensions, so here we don't care much
    output_last_dim = output.shape[-1]
    bias_flops = output_last_dim if module.bias is not None else 0
    module.__flops__ += int(np.prod(input.shape) * output_last_dim + bias_flops) # bug in this line !!!!!!!!!!!!!!!!!!!!!!!

The default type of np.prod(input.shape) is np.int32, so I change it to np.prod(input.shape,dtype=np.int64). Then, the bug was fixed

[sovrasov]

@lgylylyct thanks for catching this, I'll apply the fix!