When using the Chat interface streaming mode, does the continuously output content have no line breaks, resulting in only one line of content?

[git102347501]

When using the Chat interface streaming mode, does the continuously output content have no line breaks, resulting in only one line of content?

var completionResult = sdk.ChatCompletion.CreateCompletionAsStream(new ChatCompletionCreateRequest
{
    Messages = new List<ChatMessage>
    {
        new(StaticValues.ChatMessageRoles.System, "You are a helpful assistant."),
        new(StaticValues.ChatMessageRoles.User, "Who won the world series in 2020?"),
        new(StaticValues.ChatMessageRoles.System, "The Los Angeles Dodgers won the World Series in 2020."),
        new(StaticValues.ChatMessageRoles.User, "Tell me a story about The Los Angeles Dodgers")
    },
    Model = Models.ChatGpt3_5Turbo,
    MaxTokens = 150//optional
});

await foreach (var completion in completionResult)
{
    if (completion.Successful)
    {
        Console.Write(completion.Choices.First().Message.Content);
// The output here does not have a line break : \n\n
    }
    else
    {
        if (completion.Error == null)
        {
            throw new Exception("Unknown Error");
        }

        Console.WriteLine($"{completion.Error.Code}: {completion.Error.Message}");
    }
}

In normal mode, there will be n n similar line breaks

xxxx \n\n xxxxx

[gekah]

There will be linefeeds for paragraphs - below is the output for "tell me a story": As you can see, the line length is rather large, but there will be line breaks in the streamed fragments. You may want to add a linefeed when the role changes though. Can you post your "normal mode" sample - I can't really tell a difference between the two.

"Once upon a time, in a faraway kingdom, there lived a young prince named Alexander. He was born with a gift - he could speak to animals. His father, the king, wanted to keep this gift a secret so that no one would think his son was strange or different.

One day, the kingdom was attacked by a fierce dragon who breathed fire and destroyed everything in his path. The king's army was no match for the dragon, and the people of the kingdom were terrified.

Prince Alexander knew that he had to do something to save his people from the dragon. So he set out on a journey to find the dragon's lair.

On his way, he met many animals, including a wise old owl, a friendly deer, and a clever fox. They all offered to help him in his quest to defeat the dragon.

When Prince Alexander finally reached the dragon's lair, he found the beast sleeping soundly. He knew that he had to act quickly. He crept up to the dragon and whispered in his ear, using his gift to communicate with the creature.

To everyone's surprise, the dragon woke up and listened as the prince spoke to him. Alexander learned that the dragon was not evil, but had been forced from his home by the actions of the people in the kingdom.

The prince promised to help the dragon and together they worked out a plan to bring peace to the kingdom. The dragon agreed to leave and stop his attacks on the kingdom, and the prince promised to work with his father to help all creatures, both humans and animals, live in harmony.

From that day forward, Prince Alexander was known as a kind and wise ruler who was loved by all in the kingdom, including the animals he had helped."

[gekah]

Should have added that I used a MaxTokens of 500 for that one.

[git102347501]

There will be linefeeds for paragraphs - below is the output for "tell me a story":

As you can see, the line length is rather large, but there will be line breaks in the streamed fragments.

You may want to add a linefeed when the role changes though. Can you post your "normal mode" sample - I can't really tell a difference between the two.

"Once upon a time, in a faraway kingdom, there lived a young prince named Alexander. He was born with a gift - he could speak to animals. His father, the king, wanted to keep this gift a secret so that no one would think his son was strange or different.

One day, the kingdom was attacked by a fierce dragon who breathed fire and destroyed everything in his path. The king's army was no match for the dragon, and the people of the kingdom were terrified.

Prince Alexander knew that he had to do something to save his people from the dragon. So he set out on a journey to find the dragon's lair.

On his way, he met many animals, including a wise old owl, a friendly deer, and a clever fox. They all offered to help him in his quest to defeat the dragon.

When Prince Alexander finally reached the dragon's lair, he found the beast sleeping soundly. He knew that he had to act quickly. He crept up to the dragon and whispered in his ear, using his gift to communicate with the creature.

To everyone's surprise, the dragon woke up and listened as the prince spoke to him. Alexander learned that the dragon was not evil, but had been forced from his home by the actions of the people in the kingdom.

The prince promised to help the dragon and together they worked out a plan to bring peace to the kingdom. The dragon agreed to leave and stop his attacks on the kingdom, and the prince promised to work with his father to help all creatures, both humans and animals, live in harmony.

From that day forward, Prince Alexander was known as a kind and wise ruler who was loved by all in the kingdom, including the animals he had helped."

Will appear in the reply content \n ？

[gekah]

Yes. N.B.: Some Windows controls (e.g. RichTextBox) need \n replaced with \r\n or they will not display the line break. But the Console.Write from your sample should work with both AFAIK.

[git102347501]

Yes. N.B.: Some Windows controls (e.g. RichTextBox) need \n replaced with \r\n or they will not display the line break. But the Console.Write from your sample should work with both AFAIK.

Due to the need to set the proxy for httpclient, I directly copied the following code of the project:

        /// <summary>
        /// CreateCompletionAsStream
        /// </summary>
        /// <param name="createCompletionRequest"></param>
        /// <param name="modelId"></param>
        /// <param name="cancellationToken"></param>
        /// <returns></returns>
        public async IAsyncEnumerable<ChatCompletionCreateResponse> CreateCompletionAsStream(Guid id, string? modelId = null,
        [EnumeratorCancellation] CancellationToken cancellationToken = default)
        {
            // mycode, get by parm id data result data.Data
            var chatCompletionCreateRequest = data.Data;
            chatCompletionCreateRequest.Stream = true;

            // Send the request to the CompletionCreate endpoint
            chatCompletionCreateRequest.ProcessModelId(modelId, "");
            var handler = new HttpClientHandler()
                {
                    Proxy = new WebProxy("http://localhost:7890"),
                    UseProxy = true
                };
             var _httpClient = new HttpClient(handler);
            using var response = _httpClient.PostAsStreamAsync("https://api.openai.com/v1/chat/completions", 
                chatCompletionCreateRequest, cancellationToken);
            await using var stream = await response.Content.ReadAsStreamAsync(cancellationToken);
            using var reader = new StreamReader(stream);
            // Continuously read the stream until the end of it
            while (!reader.EndOfStream)
            {
                cancellationToken.ThrowIfCancellationRequested();

                var line = await reader.ReadLineAsync();
                // Skip empty lines
                if (string.IsNullOrEmpty(line))
                {
                    continue;
                }

                line = line.RemoveIfStartWith("data: ");

                // Exit the loop if the stream is done
                if (line.StartsWith("[DONE]"))
                {
                    break;
                }

                ChatCompletionCreateResponse? block;
                try
                {
                    // When the response is good, each line is a serializable CompletionCreateRequest
                    block = System.Text.Json.JsonSerializer.Deserialize<ChatCompletionCreateResponse>(line);
                }
                catch (Exception)
                {
                    // When the API returns an error, it does not come back as a block, it returns a single character of text ("{").
                    // In this instance, read through the rest of the response, which should be a complete object to parse.
                    line += await reader.ReadToEndAsync();
                    block = System.Text.Json.JsonSerializer.Deserialize<ChatCompletionCreateResponse>(line);
                }

                if (null != block)
                {
                    yield return block;
                }
            }
        }

Use Controller Code:

 var response = Response;
response.Headers.Add("Content-Type", "text/event-stream");
var completionResult = CreateCompletionAsStream(id);
                await foreach (var completion in completionResult)
                {
                    string res;
                    if (completion.Successful && completion.Choices.Count > 0)
                    {
                        res = completion.Choices.First().Message.Content;
                    }
                    else
                    {
                        if (completion.Error == null)
                        {
                            res = "Unknown Error";
                        }
                        else
                        {
                            res = $"{completion.Error.Code}: {completion.Error.Message}";
                        }
                    }
                    if (string.IsNullOrWhiteSpace(res))
                    {
                        continue;
                    }
                    await Response.WriteAsync($"data: {res}\r\r");
                }
                await response.Body.FlushAsync();

I guess it's due to missing parameters? Resulting in a result that I actually received without a line break similar to \n

[gekah]

Here's an idea: in your Controller, after this line: res = completion.Choices.First().Message.Content;

insert the following for testing purposes only. res = BitConverter.ToString(Encoding.UTF8.GetBytes(res))+ " ";

This will cause the received fragments to be sent to your consumer (browser?) in Hex - fragments are separated by a blank, contiguous bytes by dashes. If you see "0A", the linefeeds are there and your consumer is somehow suppressing them. No "0A" means you have to find out why they do not reach your server.

[git102347501]

Here's an idea: in your Controller, after this line: res = completion.Choices.First().Message.Content;

insert the following for testing purposes only. res = BitConverter.ToString(Encoding.UTF8.GetBytes(res))+ " ";

This will cause the received fragments to be sent to your consumer (browser?) in Hex - fragments are separated by a blank, contiguous bytes by dashes. If you see "0A", the linefeeds are there and your consumer is somehow suppressing them. No "0A" means you have to find out why they do not reach your server.

I saw 0A after converting hexadecimal code

And there are line breaks within the string

The reason is that when eventsource sends a string, it automatically truncates the newline character and no longer continues to send the information after the newline character, resulting in the front-end not receiving the newline character and its subsequent content.

I changed the transmission method to base64 byte transmission, and the front-end converted to base64 byte transmission. The problem was resolved