Somethings about llama-index evaluation

[aagq]

Could you tell me which dataset in your llama-index evaluation or how to do this, thanks

[545999961]

1. The dataset is the paper 《Llama 2: Open Foundation and Fine-Tuned Chat Models》.
2. If you want to evaluate, you need to modify the "SentenceTransformerRerank" class in llama_index, you can use "LLMReranker" class in FlagEmbedding to modify it.

[aagq]

Another question: Do you use Anthropic LLM to generate Question-Context Pairs?

[545999961]

We don't use Anthropic LLM to generate Question-Context Pairs.

[aagq]

So, how do you generate Question-Context Pairs?

[545999961]

We use GPT-3.5 to generate Question-Context Pairs.

[aagq]

How many Question-Context pairs did you generate?

[545999961]

Following llama_index, we use Pages from start to 36 for the experiment which excludes table of contents, references, and appendix.