The application now supports two methods of retrieving responses from chatCompletion. The first method allows users to export the response to an output file using the -o/--output flag. The second method prints the entire response to the console at once.
In this new feature, the goal is to implement real-time streaming of responses directly to the console (stdout) rather than outputting everything at once.
Implementation
Introduce a new argument option -s/--stream.
Use the for await ... of loop to asynchronously iterate over the chatCompletion response stream, handling each data chunk and streaming it to stdout in real-time.
Track token usage throughout the process.
Notes
Since the response will be streamed, the method of tracking token usage differs from the non-streaming approach.
For streaming, the token usage object can only be accessed after processing the final chunk in the loop.
By default, when using the -s/--stream flag without specifying an output file via -o/--output, the response will be streamed and displayed in the console in real time. However, if the user wants to export the response to a file, they can specify the output file using the -o/--output flag.
AnhChienVu
commented
1 week ago
Feature
The application now supports two methods of retrieving responses from chatCompletion. The first method allows users to export the response to an output file using the -o/--output flag. The second method prints the entire response to the console at once.
In this new feature, the goal is to implement real-time streaming of responses directly to the console (stdout) rather than outputting everything at once.
Implementation
for await ... ofloop to asynchronously iterate over the chatCompletion response stream, handling each data chunk and streaming it to stdout in real-time.Notes
-s/--streamflag without specifying an output file via-o/--output, the response will be streamed and displayed in the console in real time. However, if the user wants to export the response to a file, they can specify the output file using the-o/--outputflag.