Parameter to MergeFrom() must be instance of same class

[jmugan]

I get

Traceback (most recent call last):
  File "/home/jmugan/Dropbox/consulting/Billenote/billenote/src/experimental/gale_langchain_vertex_ai.py", line 8, in <module>
    structured_llm = llm.with_structured_output(AnswerWithJustification)
  File "/home/jmugan/anaconda3/lib/python3.8/site-packages/langchain_google_vertexai/chat_models.py", line 1101, in with_structured_output
    llm = self.bind_tools([schema], tool_choice=self._is_gemini_advanced)
  File "/home/jmugan/anaconda3/lib/python3.8/site-packages/langchain_google_vertexai/chat_models.py", line 1138, in bind_tools
    vertexai_tool = _format_to_gapic_tool(tools)
  File "/home/jmugan/anaconda3/lib/python3.8/site-packages/langchain_google_vertexai/functions_utils.py", line 231, in _format_to_gapic_tool
    gapic_tool.function_declarations.append(fd)
TypeError: Parameter to MergeFrom() must be instance of same class: expected google.cloud.aiplatform.v1beta1.FunctionDeclaration got FunctionDeclaration.

from

from langchain_google_vertexai import ChatVertexAI
class AnswerWithJustification(BaseModel):
    '''An answer to the user question along with justification for the answer.'''
    answer: str
    justification: str
llm = ChatVertexAI(model="gemini-pro", temperature=0)  
structured_llm = llm.with_structured_output(AnswerWithJustification)
response = structured_llm.invoke("What weighs more a pound of bricks or a pound of feathers")  
print(response.content)

Probably a version mismatch with protobuf silliness. I did pip install -U on everything I could think of. Any suggestions?

[alx13]

Hey @jmugan, I just did a clean install in the new venv and can't replicate this issue:

python3 -m venv venv
source venv/bin/activate
pip install -U langchain-google-vertexai

python3 240.py
answer='A pound of bricks and a pound of feathers weigh the same.' justification='They both weigh one pound.'

Also, there are two things that could be improved:

1. llm.with_structured_output(AnswerWithJustification).invoke("text") will return you the instance of AnswerWithJustification, so response.content will fail, as AnswerWithJustification doesn't have content field.
2. It's always better to use explicit model version like gemini-1.0-pro-002 https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versioning#stable-version

So the code will look like this:

from langchain_google_vertexai import ChatVertexAI
from langchain_core.pydantic_v1 import BaseModel

class AnswerWithJustification(BaseModel):
    '''An answer to the user question along with justification for the answer.'''
    answer: str
    justification: str
llm = ChatVertexAI(model="gemini-1.0-pro-002", temperature=0)
structured_llm = llm.with_structured_output(AnswerWithJustification)
response = structured_llm.invoke("What weighs more a pound of bricks or a pound of feathers")
print(response)

[jmugan]

Thanks for your response. That code gives me the same error. In a new environment, it runs with no problem, but I'm trying to use it in an existing environment. So there must be something with the compiling of the protobuf. It needs to be recompiled somehow with a new version of some library, but I can't figure out which one. I don't know why Google loves protobuf so much.

[lkuligin]

Maybe try pip uninstall libraries in question (incl. google-cloud-aiplatform) from your env first and then install them again. Closing it for now, since it doesn't look the issue is on the integration side.