Stream System

[mazinn444]

Bug description

The Streaming system doesn't work very well. Therefore, I would like you to fix this bug.

Like, if you try to run a:

for chunk in g4f.ChatCompletion(
     model="gpt-4o",
     messages=[{'role': 'user', 'content': 'Example'}],
     stream=True
):
     print(chunk, end='', flush=True)

just returns the entire response at once, no chunk by chunk

Environment

• Python 3.13.

[kqlio67]

Hi @mazinn444,

I noticed you might be using an outdated method for streaming. The documentation has been updated with the correct way to implement streaming. Here are two examples that should help you get it working properly:

Using regular client: Documentation: https://github.com/xtekky/gpt4free/blob/main/docs/client.md#streaming-completions

from g4f.client import Client

client = Client()

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "user",
            "content": "Say this is a test"
        }
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content or "", end="")

Using async client: Documentation: https://github.com/xtekky/gpt4free/blob/main/docs/async_client.md#streaming-completions

import asyncio
from g4f.client import Client

async def main():
    client = Client()

    stream = await client.chat.completions.async_create(
        model="gpt-4o",
        messages=[
            {
                "role": "user",
                "content": "Say this is a test"
            }
        ],
        stream=True,
    )

    async for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")

asyncio.run(main())

These examples use the new client implementation which properly handles streaming.

Also, make sure you're using the latest version of the library, as the streaming implementation has been improved in recent updates.

[mazinn444]

Thanks, bro. I was trying the previous method. One question, the stream system sometimes bugs, is this normal? Another question, what is the fastest gpt4 provider?