Can you say more on memory usage eg is it useful with 24G?

mustafaaljadery / gemma-2B-10M

Gemma 2B with 10M context length using Infini-attention.

933 stars 57 forks source link

Can you say more on memory usage eg is it useful with 24G? #9

Open danbri opened 3 months ago

danbri commented 3 months ago

For local hardware usage 24G is an interesting number (NVIDIA 3090 etc.). Can you give some idea on what folks might expect from this code on such hardware?