Can I limit the context window to say like 100k?

mustafaaljadery / gemma-2B-10M

Gemma 2B with 10M context length using Infini-attention.

933 stars 57 forks source link

Can I limit the context window to say like 100k? #7

Open happy15 opened 3 months ago

happy15 commented 3 months ago

Hi, really exciting to see 10M context window. But I don't have 32G memory. Can I limit the context window to 100k to reduce the required memory to be fit in 16G? I don't want to crash and restart my laptop since so many work going on. Thanks.