meta-llama / llama-stack

Model components of the Llama Stack APIs
MIT License
4.34k stars 553 forks source link

Query generators for RAG query #54

Closed hardikjshah closed 1 month ago

hardikjshah commented 1 month ago

Sometimes when doing RAG, we need custom logic to convert the user messages to a query that can be sent to memory banks for retrieval purposes. This could be as simple as strong concatenation in some cases or using an LLM for query generation.

This PR introduces a notion of MemoryQueryGenerators and associated configs. Agent reads this config to understand how to generate the query before calling memory apis.

Tested with --

  1. Start distro
    llama stack run local --name {distro_name} 
  2. Run RAG script
    PYTHONPATH=. python examples/scripts/rag_with_memory_bank.py localhost 5000 --disable-safety 
hardikjshah commented 1 month ago

Done both.

TODOs: