Closed iyuhang closed 1 month ago
Since you are trying to call the async client, you need to call it inside an async loo. Try encapsulating the call to async_response = client.chat_stream(model=model, messages=messages)
and the for loop after inside an async function, then call it using asyncio.run(my_async_func())
Hmm.. I am getting this same error when using the async client to do function calling. Works just fine up until a function response is sent.. then throws this httpx.ResponseNotRead error. Specifically this happens when I'm trying to iterate over the response using an async for ...
loop.
I am running inside an async function.
Which version of mistralai
are you using? (pip show mistralai
)
I am able to run the following code snippet below with no errors on v0.0.11 through to v0.1.6.
import asyncio
import os
from mistralai.async_client import MistralAsyncClient
from mistralai.models.chat_completion import ChatMessage
api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-large-latest"
client = MistralAsyncClient(api_key=api_key)
messages = [ChatMessage(role="user", content="What is the best French cheese?")]
# With async
async_response = client.chat_stream(model=model, messages=messages)
async def main():
async for chunk in async_response:
print(chunk.choices[0].delta.content)
# Run the async function
asyncio.run(main())
I was using latest, but it turned out that there was an error in my message syntax that was being bubbled up this way. The error message I received was a bit of a red herring.
Please let us know if you still see issues!
I use python client
use the code from document , use stream and async
It raise :
httpx.ResponseNotRead: Attempted to access streaming response content, without having called
read().