search

explodinggradients / ragas

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines
https://docs.ragas.io
Apache License 2.0
6.58k stars 646 forks source link

I've been encountering an issue where Ragas generates test cases from the initially uploaded documents, even after updating the documents and rerunning the process. It either uses the old documents or mixes them with the new ones. #1002

Open AliHaider0343 opened 3 months ago

AliHaider0343 commented 3 months ago

[ ] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question what is unclear to you? What would you like to know?

Code Examples This community speaks code. Share your code snippets to help us understand your question better.

Additional context Anything else you want to share with us?

Code

generator = TestsetGenerator.from_langchain( generator_llm, critic_llm, embeddings )

testset = generator.generate_with_langchain_docs( documents, test_size=3, distributions={ 'simple': 0.5, 'reasoning': 0.25, 'multi_context': 0.25 } )

Nandakishore-Thekkadathu commented 3 months ago

I had this same issue but it got solved after dividing the whole code into main and worker scripts.

jstamandtech commented 3 months ago

I'm also running into this issue