Any plan to support Llama 405B (Llama 3.1)

SafeAILab / EAGLE

Official Implementation of EAGLE-1 and EAGLE-2

https://arxiv.org/pdf/2406.16858

Apache License 2.0

737 stars 74 forks source link

Any plan to support Llama 405B (Llama 3.1) #103

Closed yanjunplay closed 1 month ago

yanjunplay commented 1 month ago

I've been waiting for a while, haha..

Today's the Llama3.1 405B model is officially announced https://llama.meta.com/. Do we have any plan to support this soon? I assume we need to adjust Eagle model accordingly.

Thanks a lot!!

hongyanz commented 1 month ago

As an academic group, we don't have enough GPUs to proceed the inference/training on LLama 405B (which should require 8x H100s for inference). If any industry would like to support us, please do let us know. Thanks.

yanjunplay commented 1 month ago

Thanks @hongyanz for the reply. Let me do some study and see how we can collaborate.

anilmurty commented 1 month ago

hey @hongyanz - I'd be interested in seeing if we can support you training 405B on https://akash.network/. We're able to run Llama-3.1 405B for inference. Are you interested in discussing this?