Blog: License to Call: Introducing Transformers Agents 2.0

[KannamSridharKumar]

The question is regarding - https://huggingface.co/blog/agents#self-correcting-retrieval-augmented-generation

The sources of docs in the vector index are ['blog', 'optimum', 'datasets-server', 'datasets', 'transformers', 'course', 'gradio', 'diffusers', 'evaluate', 'deep-rl-class', 'peft', 'hf-endpoints-documentation', 'pytorch-image-models', 'hub-docs']

For the query "Please show me a LORA finetuning script", the output shown in the blog shows that the agent first started with these 3 sources ['transformers', 'datasets-server', 'datasets'] and then used all sources as it couldn't find relevant context in those 3 sources.

How/why did the agent chose those 3 data sources? Were those sources picked by up the agent itself in the first pass or were they manually set by the human?

Something doesn't look right?

Thanks,

[aymeric-roucher]

The sources were the ones picked by the agent itself! So they were decided based on an LLM call.

[KannamSridharKumar]

Many thanks for the quick response.

Under the hood, is the agent computing some sort of semantic similarity between the query and all the data sources (names) to find out the potential data sources to search for context in later steps?

[aymeric-roucher]

@KannamSridharKumar: Here the agent only has access to its system prompt (which you can access by printing agent.system_prompt). In that particular agent's system prompt, the tool description have been inserted upon initialization. And here the tool description for the retriever contains the list of the databases : so the LLMs just picks from them the ones that seem useful for that query. It's still very rudimentary, it could be improved by providing more description for each of these databases.