Explore Landmark Attention

[abrichr]

https://github.com/epfml/landmark-attention

https://arxiv.org/abs/2305.16300

Our approach seamlessly integrates with specialized data structures and the system's memory hierarchy, enabling processing of arbitrarily long context lengths. We demonstrate that our method can obtain comparable performance with Transformer-XL while significantly reducing the number of retrieved tokens in each step. Finally, we show that fine-tuning LLaMA 7B with our method successfully extends its context length capacity up to 32k tokens, allowing for inference at the context lengths of GPT-4.

[FFFiend]

Models with Landmark Attention

[abrichr]

TODO: pick one or more models we want to deploy