explodinggradients / ragas

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines
https://docs.ragas.io
Apache License 2.0
6.47k stars 629 forks source link

Buggy tutorials #1075

Open amr-cloudforce opened 2 months ago

amr-cloudforce commented 2 months ago

[ X] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug A clear and concise description of what the bug is.

This tutorial doesn't seem to be functioning properly. I question whether the developer has actually run this code. Even the class names aren't accurate, like 'OpenAIEmbedding' for instance.

Among others:

https://docs.ragas.io/en/latest/howtos/applications/compare_embeddings.html

It would have been beneficial to provide a sample file with the necessary libraries for execution beforehand.

Ragas version: Python version:

Code to Reproduce Share code to reproduce the issue

Error trace

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

dosubot[bot] commented 2 months ago

Hello @amr-cloudforce! I'm here to help you with any bugs, questions, or contributions you might have. Let's work together to resolve the issue you're facing.

The tutorial on comparing and choosing embeddings for data using the Ragas library does not function properly because the class names like OpenAIEmbedding are inaccurate. The correct class name should be OpenAIEmbeddings.

Incorrect usage in the tutorial:

openai_model = OpenAIEmbedding()

Correct usage should be:

openai_model = OpenAIEmbeddings()

Additionally, the Ragas library uses a wrapper class LangchainEmbeddingsWrapper to handle embeddings, which should be instantiated correctly. Here is an example of how to properly set up the embeddings using the Ragas library:

from ragas.embeddings.base import LangchainEmbeddingsWrapper
from langchain_openai.embeddings import OpenAIEmbeddings

openai_model = OpenAIEmbeddings(model="text-embedding-ada-002")
wrapped_model = LangchainEmbeddingsWrapper(openai_model)

This ensures that the embeddings are correctly instantiated and wrapped for use within the Ragas framework [1][2].

To continue talking to Dosu, mention @dosu.

jjmachan commented 1 month ago

@amr-cloudforce thanks for pointing it out! would you like to take a hand at contributing a fix? would really appreciate the help and I can help you out as well 🙂