Generate a Synthetic Test Set Program has issue

[Kunal56377]

[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug

I have used the code to generate Synthetic Test set for my document. But Somehow it is going to infinite loop

Ragas version: latest Python version: 3.11.0

import os from langchain_community.document_loaders import DirectoryLoader from ragas.testset.generator import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context from langchain_openai import ChatOpenAI, OpenAIEmbeddings from ragas.run_config import RunConfig

os.environ["OPENAI_API_KEY"] = "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"

loader = DirectoryLoader("docs\Ltsa_Help_rags") documents = loader.load()

for document in documents: document.metadata['filename'] = document.metadata['source'] print(document.metadata['filename'] )

generator_llm = ChatOpenAI(model="gpt-3.5-turbo-16k") critic_llm = ChatOpenAI(model="gpt-4") embeddings = OpenAIEmbeddings() print(generator_llm.invoke("Hi How are you?")) generator = TestsetGenerator.from_langchain( generator_llm, critic_llm, embeddings ) k = RunConfig() k.max_retries = 1 k.max_workers = 1 k.timeout = 10 k.max_wait = 10 print(k) testset = generator.generate_with_langchain_docs(documents, test_size=2,distributions={simple: 1.0},with_debugging_logs=True,run_config = k)

df = testset.to_pandas() df.to_csv("Sytentic_test.csv")

Error trace

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

[shahules786]

Hey @Kunal56377 , the problem here is that the node_filter giving score less than required threshold which is 7.5. Would recommend modifying node_filter to suit your needs ( 7 or less) . You can see the parameter here