AttributeError: 'str' object has no attribute 'content'

[Rugved2204]

❗BEFORE YOU BEGIN❗ Are you on discord? 🤗 We'd love to have you asking questions on discord instead: https://discord.com/invite/a3K9c8GRGt

Describe the bug AttributeError: 'str' object has no attribute 'content'

To Reproduce class AWSBedrock(DeepEvalBaseLLM): def init( self, model ): self.model = model

def load_model(self):
    return self.model

def generate(self, prompt: str) -> str:
    chat_model = self.load_model()
    return chat_model.invoke(prompt).content

async def a_generate(self, prompt: str) -> str:
    chat_model = self.load_model()
    res = await chat_model.ainvoke(prompt)
    print(res)
    return res.content

def get_model_name(self):
    return ""

llm = BedrockLLM(model_id='mistral.mistral-large-2402-v1:0', client=bedrock,streaming=True)

aws_bedrock = AWSBedrock(model=llm)

Expected behavior A clear and concise description of what you expected to happen.

Traceback Traceback (most recent call last): File "/home/ubuntu/scp-analyzer/rag-old/dpeval.py", line 185, in synthesizer.generate_goldens_from_docs( File "/home/ubuntu/.local/lib/python3.10/site-packages/deepeval/synthesizer/synthesizer.py", line 900, in generate_goldens_from_docs return loop.run_until_complete( File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/home/ubuntu/.local/lib/python3.10/site-packages/deepeval/synthesizer/synthesizer.py", line 860, in a_generate_goldens_from_docs return await self.a_generate_goldens( File "/home/ubuntu/.local/lib/python3.10/site-packages/deepeval/synthesizer/synthesizer.py", line 676, in a_generate_goldens await asyncio.gather(*tasks) File "/home/ubuntu/.local/lib/python3.10/site-packages/deepeval/synthesizer/synthesizer.py", line 223, in _a_generate_fromcontexts res, = await self.a_generate(prompt) File "/home/ubuntu/.local/lib/python3.10/site-packages/deepeval/synthesizer/synthesizer.py", line 91, in a_generate return (await self.model.a_generate(prompt), 0) File "/home/ubuntu/scp-analyzer/rag-old/dpeval.py", line 143, in a_generate return res.content AttributeError: 'str' object has no attribute 'content'

Desktop (please complete the following information): python:3.10 deepeval:0.21.71

Additional context Add any other context about the problem here.

[darioxz]

This issue is difficult to read and lacks details. Anyway, I'll try to help you.

It seems to me that the model you are trying to use does not return the expected type.

Please have a look at the following AWS Bedrock example: https://github.com/confident-ai/deepeval/blob/main/docs/docs/metrics-introduction.mdx#aws-bedrock-example

It seems to me that this example is not correct, though: The ainvoke method seems to return a str, see https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/language_models/llms.py#L360-L379

So in your existing code, you might just change the line return res.content to return res as the result already is the generated string.

Let me know if that works.