Open abetatos opened 1 month ago
I forgot to add that the error was tested both in a jupyter notebook (using nest_asyncio.apply()) and a .py.
And the imports were both tested with
from langchain_google_vertexai import VertexAIEmbeddings, VertexAI
and
from langchain.chat_models import ChatVertexAI
from langchain.embeddings import VertexAIEmbeddings
Same problem for me with VertexAI stack in jupyter notebook. Slightly different models and embeddings used, but I'm getting this error right from the start of test set generation process.
@abetatos UPD
I've take a look into this PR, as a result of issue you've mentioned here.
I tried to clone that dev branch with fix, and install ragas from that source, but that hasn't worked. Then, I tried just to replace the src/ragas/executor.py file contents with the fixed code, presented in that PR, in my local (master) ragas package, and that worked - I'm able to generate test datasets with the VertexAI stack.
I still don't understand what is the root cause, and whether that fix is legit (since repo author hasn't really answered and merged that PR), but at least that works.
This is related to: https://github.com/explodinggradients/ragas/issues/681
Describe the bug TestGeneration launches an error with default example of https://docs.ragas.io/en/stable/concepts/testset_generation.html with VertexAI models.
Ragas version: 0.1.7 Python version: Python 3.10.13
Code to Reproduce
Error trace
If you need any additional info please let me know! Thank you very much.