`Flux<String>` (or `.stream().content()`) trims left whitespace

[seguri]

Hi everybody. I started building my own AI chat bot with ollama while reading "Spring AI in Action". I decided to implement something I use often, i.e. translating text from one language to another.

My code is very simple:

@Service
class TranslationService(builder: ChatClient.Builder) {
  private val chatClient: ChatClient = builder.build()
  fun translateStream(message: String, from: String, to: String): Flux<String> {
    return chatClient.prompt().system(getTemplate(from, to)).user(message).stream().content()
  }
}

I have a REST controller that exposes this service. I tested it with curl and httpie, and they showed:

HTTP/1.1 200
Content-Type: text/event-stream

data:La
data: frase
data: "
data:come
data: st
data:ai
data:?"

I've then built a simple HTML page where I can send questions to this endpoint through an EventSource. The developer tools always showed that the leading whitespace was missing:

La
frase
"
come
st
ai
?"

After hours I stumbled upon 9.2.6 Interpreting an event stream where the last example ends with:

This is because the space after the colon is ignored if present.

I've suddenly realized that the problem was the missing space after the colon. There should be always one, and when a chunk starts with a space, there should be two. In my example, Spring Boot should return a double space and not one, i.e. data: frase. I've currently fixed this in my application code by extending StreamResponseSpec:

fun ChatClient.ChatClientRequest.StreamResponseSpec.contentWithWhitespace(): Flux<String> {
  return this.chatResponse().map {
    val content = it.result?.output?.content ?: ""
    if (content.startsWith(" ")) " $content" else content
  }
}

The project reactor-core redirected me here.

My code is here, the relevant commit is named feat: chapter 2.

Thank you for your attention.

[KAMO030]

Changing the return value to a data class solves this problem; for example Flux<Message>

fun ChatClient.StreamResponseSpec.toMessage(): Flux<Message> = chatResponse().mapNotNull {
    it?.result?.output?.let { outPut ->
        if (outPut.content.isNullOrEmpty()) return@mapNotNull null
        Message(outPut.messageType.value, outPut.content)
    }
}

[seguri]

Changing to Flux<Message> makes the EventSource receive:

data:{"role":"assistant","content":"**","images":[]}
data:{"role":"assistant","content":"Come","images":[]}
data:{"role":"assistant","content":" ti","images":[]}
data:{"role":"assistant","content":" sent","images":[]}
data:{"role":"assistant","content":"i","images":[]}
data:{"role":"assistant","content":" oggi","images":[]}
data:{"role":"assistant","content":"?","images":[]}
...

which seems to make everything more complicated than my simple contentWithWhitespace, but thanks for teaching me something new about Kotlin. I think that how Flux<String> is currently implemented is wrong considering the spec I quoted, as it sends data: rather than data:.