Lyken17 / pytorch-OpCounter

Count the MACs / FLOPs of your PyTorch model.
MIT License
4.9k stars 528 forks source link

will torch.matmul regards as zero_ops ? #213

Open DavideHe opened 1 year ago

DavideHe commented 1 year ago

when I get the macs with thop ,I find the op's macs of torch.matmul will is 0, but it is very heavy macs in self attention of transformer

class M(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self,x):
        out = torch.matmul(x,x.transpose(-1, -2))
        print(out.shape)
        return out