Is it considering output length of transformer models?

MrYxJ / calculate-flops.pytorch

The calflops is designed to calculate FLOPs、MACs and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer(Bert、LlaMA etc Large Language Model)

https://pypi.org/project/calflops/

MIT License

587 stars 21 forks source link

Is it considering output length of transformer models? #25

Closed liut89 closed 5 months ago

liut89 commented 5 months ago

I think the result FLOPs and MACs are influenced not only by input length but also output length, I'm I right? If yes, is the tool considering out put length? How to control the output length?

MrYxJ commented 5 months ago

Of course, it is relevant. The calculation of FLOPs here strictly defaults to the calculation amount of the prefill phase of the model, which can be regarded as the default output of a token.