Why is the proportion of FLOPs in each submodule always half of the proportion of MACS

MrYxJ / calculate-flops.pytorch

The calflops is designed to calculate FLOPs、MACs and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer(Bert、LlaMA etc Large Language Model)

https://pypi.org/project/calflops/

MIT License

392 stars 14 forks source link

Why is the proportion of FLOPs in each submodule always half of the proportion of MACS #7

Open wulinunu opened 9 months ago

wulinunu commented 9 months ago

Though 是FLOPs is around twice as much as MACS, isn't the proportion of each submodule supposed to be equal.

liut89 commented 3 weeks ago

I realized this too:

The 13.96 GFLOPS = 50% FLOPs should be 13.96 GFLOPS = 100% FLOPs.