Open mazinn444 opened 5 hours ago
Hi @mazinn444,
I noticed you might be using an outdated method for streaming. The documentation has been updated with the correct way to implement streaming. Here are two examples that should help you get it working properly:
Using regular client: Documentation: https://github.com/xtekky/gpt4free/blob/main/docs/client.md#streaming-completions
from g4f.client import Client
client = Client()
stream = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "user",
"content": "Say this is a test"
}
],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content or "", end="")
Using async client: Documentation: https://github.com/xtekky/gpt4free/blob/main/docs/async_client.md#streaming-completions
import asyncio
from g4f.client import Client
async def main():
client = Client()
stream = await client.chat.completions.async_create(
model="gpt-4o",
messages=[
{
"role": "user",
"content": "Say this is a test"
}
],
stream=True,
)
async for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
asyncio.run(main())
These examples use the new client implementation which properly handles streaming.
Also, make sure you're using the latest version of the library, as the streaming implementation has been improved in recent updates.
Thanks, bro. I was trying the previous method. One question, the stream system sometimes bugs, is this normal? Another question, what is the fastest gpt4 provider?
Bug description
The Streaming system doesn't work very well. Therefore, I would like you to fix this bug.
Like, if you try to run a:
just returns the entire response at once, no chunk by chunk
Environment