how to do doc cache if there are multiple documents in rag?

ContextualAI / gritlm

Generative Representational Instruction Tuning

https://arxiv.org/abs/2402.09906

MIT License

538 stars 39 forks source link

how to do doc cache if there are multiple documents in rag? #3

Open SimonYx42 opened 7 months ago

SimonYx42 commented 7 months ago

generally, we will have several docs for reference while doing rag, wondering if you guys have tested this setting with doc cache?

SimonYx42 commented 7 months ago

and also, is it possible to share the rag code and setting? really appreciate it!

Muennighoff commented 7 months ago

Hmm we havn't tested it, but I think it could work; Similar to Query-Doc / Doc-Query, you can just concatenate the key-value states; The problem will be the attention mismatch, but still worth trying I think!

Sure all the RAG code is in https://github.com/ContextualAI/gritlm/tree/main/rag