search

openlm-research / open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset
Apache License 2.0
7.29k stars 372 forks source link

longer context #25

Closed ehartford closed 1 year ago

ehartford commented 1 year ago

I know that you are trying to reproduce llama architecture - but, is there a way to do that but with a change to allow for longer context? we really need 8k context, if not 32k. 2k is too limiting.

ehartford commented 1 year ago

duplicate of https://github.com/openlm-research/open_llama/issues/3