Closed vfan26 closed 7 months ago
It seems that the thop library does not calculate FLOPs for matrix multiplication used in XCA module. There is a simple example.
import torch import torch.nn as nn from thop import profile, clever_format class MatMul(nn.Module): def __init__(self): super(MatMul, self).__init__() def forward(self, a, b): return a @ b matmul = MatMul().cuda() a = torch.randn(1, 1, 100).cuda() b = torch.randn(1, 100, 1).cuda() macs, params = profile(matmul, inputs=(a, b)) macs, params = clever_format([macs, params], "%.3f") print(macs, params)
output
0.000B 0.000B
Hi! Thanks for your findings! I have just tested with your code. Indeed, thop doesn't count matmul operations.
thop
matmul
It seems that the thop library does not calculate FLOPs for matrix multiplication used in XCA module. There is a simple example.
output