Set the context size for LLama3

Lightning-AI / litgpt

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

https://lightning.ai

Apache License 2.0

6.95k stars 733 forks source link

Closed awaelchli closed 1 month ago

awaelchli commented 1 month ago

Sets the context size for Llama3. The model card reports it is 8k. I guess it is probably a power of 2. https://huggingface.co/meta-llama/Meta-Llama-3-70B

Thanks @IvanYashchuk