Crashes when trying to stream Chat Vertex AI

thehunmonkgroup commented 10 months ago

System Info

Langchain v0.0.298 Python 3.11.4 Pop!OS 22.04

Who can help?

@hwchase17 @agol

Information

[X] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[X] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction

When trying to stream chat-bison:

  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/base.py", line 204, in stream
    for chunk in self._stream(
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/vertexai.py", line 195, in _stream
    responses = chat.send_message_streaming(question.content, **params)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ChatSession' object has no attribute 'send_message_streaming'

As far as I can tell, there is no send_message_streaming() method declared anywhere in Langchain.

When tryign to stream codechat-bison:

  File "/home/hunmonk/git/llm-workflow-engine/lwe/backends/api/request.py", line 320, in iterate_streaming_response
    for chunk in self.llm.stream(messages):
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/base.py", line 215, in stream
    raise e
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/base.py", line 204, in stream
    for chunk in self._stream(
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/vertexai.py", line 194, in _stream
    chat = self._start_chat(history, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/vertexai.py", line 210, in _start_chat
    return self.client.start_chat(message_history=history.history, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: CodeChatModel.start_chat() got an unexpected keyword argument 'message_history'

That looks like a Pydantic validation error in the class itself?

Expected behavior

Streaming these models should work.

dosubot[bot] commented 10 months ago

🤖

Hello,

Thank you for bringing this to our attention. It seems like there are a couple of issues here.

The first error you're encountering, 'ChatSession' object has no attribute 'send_message_streaming', is likely due to the ChatSession object not having a method named send_message_streaming. This method is being called in the _stream method of the ChatVertexAI class. If this method does not exist in the ChatSession object, it would indeed cause an AttributeError.

The second error, 'CodeChatModel.start_chat()' got an unexpected keyword argument 'message_history', is due to the start_chat method of the CodeChatModel class not accepting a parameter named message_history. This parameter is being passed in the _start_chat method of the ChatVertexAI class. If the start_chat method does not accept this parameter, it would cause a TypeError.

To resolve these issues, we need to ensure that the send_message_streaming method exists in the ChatSession object and the start_chat method of the CodeChatModel class accepts a parameter named message_history. If these methods do not exist or do not accept these parameters, it might be necessary to update your version of the VertexAI library or check the documentation for these classes to ensure you are using them correctly.

In the context shared, it seems that the BaseChatModel and SimpleChatModel classes do not have a send_message_streaming method or a start_chat method that accepts a message_history parameter. You may need to implement these methods in your custom chat model class or modify the ChatVertexAI class to not use these methods and parameters. The shared context also shows that the _stream, _astream, and _agenerate methods are not implemented in the BaseChatModel class. These methods need to be implemented in your custom chat model class.

I hope this helps. If you have any further questions or need additional clarification, please don't hesitate to ask.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

thehunmonkgroup commented 10 months ago

Updating google-cloud-aiplatform package to 1.33.1 fixed the issue.

langchain-ai / langchain