Project 2: Create a RAG + Vector DB + LLM pipeline

[TobeTek]

Create a pipeline that integrates Retrieval-Augmented Generation (RAG) based on the LPI dataset, a vector database, and a Large Language Model (LLM).

• [ ] Use the vector database to provide the LLM model with context about the LPI dataset.
• [ ] Ensure the LLM can answer questions about the LPI dataset and make recommendations.
• [ ] Store the vector data (consider parquet files, QuadrantDB, SingleStore, ChromaDB, PGVector, or even MongoDB Vector DB)

[Tacoman99]

Notebook showing a example using advance rag techniques such has auto retrieval and hybrid search on the LPI Dataset using Llama Index, Gemini and weaviate vector database

Findings: Rag is able to provide the correct courses from a users query but the Gemma2b model struggles on returning a desired output as seen in the the notebook. Copying the fewshot templates inputting infomation instead of using the contextual infomation Next Steps:

• Tinker with the prompt engineering and templates
• Try the fine tune model instead of the base model

https://www.kaggle.com/code/tacoman789/gemma-few-shot-learning-with-llamaindex?scriptVersionId=202917060

[Tacoman99]

2nd version

• using Gemma 2 2b variant instead of Gemma
• More Metadata {course, source, model}
• More prompt engineering Summary: I found that Gemma tends to copy my few shot example instead of using a the contextual information from the RAG. Emptying the table, and prompting the model to fill out the template achieve better results. Furthermore using Gemma 2 instead of gemma provided better results which is expected since Gemma 2 has better reasoning capabilities

https://www.kaggle.com/code/tacoman789/gemma-few-shot-learning-with-llamaindex/notebook#Testing-a-different-prompt