This PR is created to discuss possible OpenAI\_request refactoring.
What's changed:
Collect chunks during streaming, and instead of bare text,
return synthetic response object, similar to one returned when stream: false
Simplify ChatSession\generate_response, as there is no longer a need to re-parse the response strings
Reasoning:
I argue that a simple plain-text return value proves to be quite ineffective when dealing with multiple SSE events. Consequently, it is more beneficial to allow the callback function to procure the entirety of the parsed data.
Despite this, receiving integrated data as a return value might still be quite convenient.
Use Case: The callback function primarily serves to display streaming output to the user. However, after this process, it becomes simpler to handle the integrated data.
The 1st one is most important, and I consider it as necessity. And I would like to know your opinion on the 2nd one.
To be done:
[x] Collect other properties, such as finish_reason, usage, etc.
(On top of #10)
This PR is created to discuss possible
OpenAI\_request
refactoring.What's changed:
stream: false
ChatSession\generate_response
, as there is no longer a need to re-parse the response stringsReasoning:
The 1st one is most important, and I consider it as necessity. And I would like to know your opinion on the 2nd one.
To be done:
finish_reason
,usage
, etc.