Integrate with RAG

[amztc34283]

Does the current version allow RAG lookup?

[eyurtsev]

There's a retrieval mode that you can test out -- we haven't benchmarked it.

The retrieval mode implemented in the service is only per document rather than across documents.

On a per document basis it's likely reducing recall and increasing precision -- b/c it will be identifying only a few relevant chunks. This doesn't make sense for all use cases. Probably best for use cases where there are long documents where a fact is only expected to appear once or twice overall rather than multiple times across pages.

IMO a good use case for mixing in RAG is when working with a standard set of documents so that the documents can be pre-indexed (rather than indexed at run time), and where we expect users to want to interact with them more than once. If relevant for your use case, you could test this out using the service by building out the retrieval component externally, and sending in the relevant snippets for extraction into the extraction service.

[lhotwll217]

RAG pipeline endpoint? /extract/{id/rag/{slackRAGId}/?q=collect-all-the-threads-my-coworkers-mention-ai-in-and-give-me-the-metadata-matching-the-schema - I guess ideally your RAG service would be able to route to the appropriate RAG implementation based off the query if you happened to want to integrate more than 1. Forego the id and DB work and keep the machine lean.