Add Streaming Option to DuckChat Responses

[lunaticsm]

Description:

This pull request introduces a new stream option to the DuckChat class, allowing responses to be streamed as they are received. The key changes include:

1. Streaming Option in ask_question:

   • A stream parameter has been added to the ask_question method, enabling real-time streaming of responses.
   • If stream=True, the response is printed in chunks as they arrive. If stream=False, the response is handled as before, being fully assembled before returning.

2. CLI Support for Streaming:

   • The CLI has been updated to accept a --stream flag with user input. When this flag is used, the response is streamed in real-time.

Why This Change is Necessary:

The new streaming functionality improves the responsiveness and interactivity of the bot, especially for long or complex queries that take time to process. Users can now see parts of the response immediately, making the bot feel more dynamic and engaging.

Testing:

The new streaming functionality has been tested with various queries to ensure that it behaves as expected, both in normal and streaming modes.

Documentation:

• Inline documentation has been updated to reflect the changes.
• Usage examples have been provided in the code to demonstrate the new streaming functionality.

[lunaticsm]

It's work when i use as cli or this

import asyncio
from duck_chat import DuckChat, ModelType

async def main():
    async with DuckChat(model=ModelType.GPT4o) as chat:
        print(await chat.ask_question("What is the capital of France?"))

        await chat.ask_question("Tell me a long story...", stream=True)

asyncio.run(main())

But when i tried using as lib on my bot, the bot always send me a full message and not with stream method.

[mrgick]

Hello, @lunaticsm

I refactored your code:

• there is now 3 stream func:
  • stream_answer (low, that do stream from api)
  • ask_question_stream (Stream answer from chat AI)
  • reask_question_stream (retry ask with stream)

In ClI there is now two cmds:

• stream_on (enable stream response)
• stream_off (disable stream response)

But when i tried using as lib on my bot, the bot always send me a full message and not with stream method. You used just return and not the generator in ask_question (see example below)

You can test with this

import asyncio

from duck_chat import DuckChat

async def main():
    async with DuckChat() as chat:
        async for message in chat.ask_question_stream(
            "Tell me long story (60 sentences)"
        ):
            print(message, end="", flush=True)
        await asyncio.sleep(1)
        async for message in chat.ask_question_stream(
            "Summaries the story in one sentence"
        ):
            print(message, end="", flush=True)
        await asyncio.sleep(1)
        async for message in chat.reask_question_stream(2):
            print(message, end="", flush=True)

asyncio.run(main())

If you have any questions or suggestions, feel free to tell me ^^

Have a good day =)

[mrgick]

I also created another funcs because they are async generators (use yield) and the other funcs use return

Also, write if it is okey - I will merge then

[lunaticsm]

Yup, that's fine. since i'm not on computer right now I'll try later, you can merge it. ^^