twitter / finagle

A fault tolerant, protocol-agnostic RPC system
https://twitter.github.io/finagle
Apache License 2.0
8.79k stars 1.46k forks source link

Stream writer/reader only read first element sent #900

Closed politrons closed 3 years ago

politrons commented 3 years ago

Hi all,

I'm using version 18.2.0 of Finagle.

Having a Server configure for streaming

      Http.server
        .withStreaming(enabled = true)
        .serve(s"0.0.0.0:$port", service)

When I write in the Writable with a delay between every write

         iter.foreach(n => {
            println(s"Server number:$n")
            writable.write(Buf.Utf8(n))
            Thread.sleep(500)
          })

In the reader I'm able to read all the element send

  def fromReader(reader: Reader): AsyncStream[Buf] =
    AsyncStream.fromFuture(reader.read(Int.MaxValue)).flatMap {
      case None =>
        AsyncStream.empty
      case Some(buf) =>
        println(buf)
        buf +:: fromReader(reader)
    }

BUT if I remove the thread.sleep(500) in the Server for each write, I can only see in the client that the first number is printed, but no more elements are read.

Somehow I think the emission of elements are just read in one reader.read getting the first element, and the rest is discarded.

Any idea how to configure the client/server to avoid that?

politrons commented 3 years ago

I just figure that the writable which is a future must be compose with another writable to avoid backpressure issues