logan-markewich
commented
2 weeks ago This is extremely similar to stuff like the Summary metadata extractor that we already have. The cool part is using prompt caching here
In any case, there's a notebook here for how to do this https://docs.llamaindex.ai/en/stable/examples/cookbooks/contextual_retrieval/
Feature Description
Anthropic has posted an impressive approach for reducing RAG errors by 67% and keeping costs reasonable in the process. Blog post: https://www.anthropic.com/news/contextual-retrieval Cookbook: https://github.com/anthropics/anthropic-cookbook/blob/main/skills/contextual-embeddings/guide.ipynb
Reason
To implement this in llamaindex would require some crafty adaptation into a Node Parser I’m guessing.. maybe even a whole new NP?
Value of Feature
from the anthropic blog post:
“ This method can reduce the number of failed retrievals by 49% and, when combined with reranking, by 67%. These represent significant improvements in retrieval accuracy, which directly translates to better performance in downstream tasks “