search

badripatro / simba

Simba
156 stars 15 forks source link

About Flops #3

Closed AZZMM closed 2 months ago

AZZMM commented 3 months ago

Hi author, thank you for sharing your excellent work!

I have a question about the FLOPs counting.

The code writes that you count the FLOPs with the repo mamba-minimal , but in mamba's issue#110 the author of mamba claimed that the actual implementation needs much more FLOPs than simply scan in a for loop.

Can you tell me what approach you have adopted in calculating the FLOPs? Also, what about the throughput of Simba?

Thank you.

badripatro commented 2 months ago

We have adopted Mamba-minimal for calculating the FLOPs.