大模型微调 - Githubissues

MrYxJ / calculate-flops.pytorch

The calflops is designed to calculate FLOPs、MACs and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer(Bert、LlaMA etc Large Language Model)

https://pypi.org/project/calflops/

MIT License

392 stars 14 forks source link

大模型微调 #1

Closed Pobby321 closed 9 months ago

Pobby321 commented 10 months ago

可以计算大模型微调时候的flops嘛

MrYxJ commented 10 months ago

可以通过设置与微调时一致的batch_size计算FLOPs。