ValidationError: 1 validation error for Prompt

[beatG123]

[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug When I try to use generator.adapt(language='chinese'), I still encounter errors related to the Prompt. I have tried multiple methods but still cannot resolve the issue.

Ragas version:0.1.18 Python version:3.10.9

Code to Reproduce from utils import * from langchain_community.document_loaders import PyMuPDFLoader

1.读取原始数据

PROJECT_PATH = os.path.dirname(file) data_path = os.path.join(PROJECT_PATH, 'data/eval_document_data/') pdf_files = [f for f in os.listdir(data_path) if f.endswith('.pdf')]

创建一个空列表来存储文档

documents = []

遍历每个 PDF 文件，并使用 PyMuPDFLoader 加载它们

for pdf_file in pdf_files: full_pdf_path = os.path.join(data_path, pdf_file) loader = PyMuPDFLoader(full_pdf_path) documents.extend(loader.load())

if len(documents) != 0: print("数据加载完成。")

print(documents)

2.数据生成

from ragas.testset.generator import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context

generator with openai models

generator_llm = ChatOpenAI(model="gpt-3.5-turbo-16k") critic_llm = ChatOpenAI(model="gpt-3.5-turbo-16k") embeddings = get_embeddings_model()

generator = TestsetGenerator.from_langchain( generator_llm, critic_llm, embeddings )

设置语言类型

language = 'chinese' generator.adapt(language=language,evolutions=[simple,reasoning,multi_context], cache_dir="a_path") generator.save(evolutions=[simple, reasoning, multi_context], cache_dir="a_path")

testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25}) print(testset.to_pandas())

Error trace raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for Prompt root output in example 1 is not in valid json format: Expecting value: line 1 column 1 (char 0) (type=value_error)

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

[dosubot[bot]]

To resolve the "ValidationError: 1 validation error for Prompt" when using generator.adapt(language='chinese') in Ragas version 0.1.18 with Python 3.10.9, ensure that all required fields in the Prompt class are properly populated. Specifically, the instruction, input_keys, and output_key fields must not be empty.

Here is an example of how to correctly initialize a Prompt object and use the adapt method:

from ragas.llms import BaseRagasLLM
from ragas.llms.prompt import Prompt

# Example LLM class (you need to implement this based on your requirements)
class MyLLM(BaseRagasLLM):
    def generate_text(self, prompt):
        # Implement the text generation logic here
        pass

# Initialize the Prompt object with all required fields
prompt = Prompt(
    name="example_prompt",
    instruction="Translate the following text to Chinese.",
    output_format_instruction="",
    examples=[
        {
            "translate_to": "chinese",
            "input": "Hello, how are you?",
            "output": "你好，你好吗？"
        }
    ],
    input_keys=["translate_to", "input"],
    output_key="output",
    output_type="str",
    language="english"
)

# Initialize your LLM
llm = MyLLM()

# Adapt the prompt to Chinese
adapted_prompt = prompt.adapt(language='chinese', llm=llm)

print(adapted_prompt)

Ensure that the instruction, input_keys, and output_key fields are not empty, as these are validated in the Prompt class. Additionally, make sure that the Prompt class and its methods, such as adapt, are correctly implemented and imported from the appropriate modules in the Ragas framework ^[1][2].

To continue talking to Dosu, mention @dosu.