max retries exceeded for SimpleEvolution

[AshuKhandave]

from ragas.testset import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context

test_generator = TestsetGenerator( generator_llm=bedrock_model, critic_llm=bedrock_model, embeddings=bedrock_embeddings, docstore=docstore, )

distributions = {simple: 0.5, reasoning: 0.4, multi_context: 0.1}

testset = test_generator.generate_with_langchain_docs( documents=documents, test_size=15, distributions=distributions )

after running this code the embeddings are created but there seems to be issue with generating the test data.

I'm trying to create the test dataset using llama2 from amazon bedrock.

[AshuKhandave]

@jjmachan could you please look into this issue

[shahules786]

Hey @AshuKhandave Sorry for the late reply. Which models/ragas version are you using? Consider joining our discord channel for quick help.

[AshuKhandave]

Hey @shahules786 I'm using meta_llama2_13b and Amazon Titan for embedding using AWS bedrock.

[shahules786]

I see @AshuKhandave would you be able to use better models? The thing with synthetic data generation is that only high-quality models like gpt 3.5 or on par with gpt 3.5 could produce high-quality results out of the box.

[MohammedAlsayed]

I'm getting the same error when trying on OpenAI