Some questions about LLM settings?

[bwnjnOEI]

1. How many LLMs are needed for setting? In your paper PaperQA: Retrieval-Augmented Generative Agent for Scientific Research, this paper seems to have employing four LLM instances,
   agent LLM, summary LLM,ask LLM and answer LLM. TheAgent LLM adjust and iterate perform steps in response to each question, ensuring more precise and relevant answers; In gather evidence module, use the summary LLM summarize chunks and evaluate relevance score on retrieved chunk; In answer question module, use the ask LLM to extract prior knowledge about the input query and use answer LLM for output the final answer. On the other side, in the example code snippet,

    settings=Settings(
        llm='ollama/qwen2.5:3b',
        llm_config=local_llm_config,
        summary_llm='ollama/qwen2.5:3b',
        summary_llm_config=local_llm_config,
        embedding='ollama/mxbai-embed-large',
        agent=AgentSettings(
            agent_llm='ollama/qwen2.5:3b', 
            agent_llm_config=local_llm_config
        ),
        paper_directory="my_papers"
    ),

imply you just need setup three LLMs: agent_llm and summary_llm started up just as the paper claimed, however,

3. what is llm? Is it ask LLM or answer LLM? Or for model reuse of ask LLM and answer LLM (where only one model is used for both)? Is there anything I missed?
4. If separate LLMs are set up (using different models), which one performs better as the strongest model? For example: setting the summary_llm to use a local 13B model while calling OpenAI or Anthropic's API for the agent_llm?

[dosubot[bot]]

Hey there, @bwnjnOEI! I'm here to help you with any bugs, questions, or contributions you might have. Let's dive into the issue you've raised.

In the Settings class, the llm parameter is used as the default language model for generating answers, which suggests it is likely used for both the ask LLM and answer LLM functionalities. This means that the llm parameter could be serving as a model reuse for both asking questions and generating answers, rather than requiring separate models for each task ^[1].

In the example code snippet you provided, it appears that three LLMs are explicitly set up: agent_llm, summary_llm, and llm. The llm parameter is likely used for both asking and answering, which aligns with the idea of model reuse for ask LLM and answer LLM.

To continue talking to Dosu, mention @dosu.