This repository contains code and tooling for the Abacus.AI LLM Context Expansion project. Also included are evaluation scripts and benchmark tasks that evaluate a model’s information retrieval capabilities with context expansion. We also include key experimental results and instructions for reproducing and building on them.
Apache License 2.0
582
stars
36
forks
source link
How do i increases the context of already fined tuned or base model of llama2 ? #8
I want to increase the context of the llama2 model, I have finetuned a model(70b and 7b) as well on my data now I want to increase their input context using yarn I was able to understand that we need data of 16k context if we want to increase the context can anyone clarify the procedures that follows after that
I want to increase the context of the llama2 model, I have finetuned a model(70b and 7b) as well on my data now I want to increase their input context using yarn I was able to understand that we need data of 16k context if we want to increase the context can anyone clarify the procedures that follows after that