Leveling up Evals: self-grading (CRAG)

[TanGentleman]

These are extremely valuable resources:

1. https://hamel.dev/blog/posts/evals/ This is helpful for several stages in the iterative process of scaling my project. A big focus on this is unit tests.
2. https://jxnl.github.io/blog/writing/2024/02/28/levels-of-complexity-rag-applications/ This is helpful for a lot of the things I need to build into my unstructured data ->structured database pipeline. For instance, it gives some great insights about why I should log the citations, as well as mean cosine scores. Also has some great bits on evaluation and other helpful bits for data handling.
3. https://hamel.dev/notes/ This is a great collection of resources from someone with a years more of experience iterating on things I'm working on.
4. Implementing CRAG and self-rag. Based on this paper: https://arxiv.org/pdf/2401.15884
   • https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_crag_local.ipynb
   • https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_self_rag_local.ipynb This is what I consider to be the most apt way for me to improve the performance of my RAG pipelines without getting too deep into knowledge graphs and creating more complex associations between my chunked documents.