softwaremill / sttp

The Scala HTTP client you always wanted!
https://sttp.softwaremill.com
Apache License 2.0
1.46k stars 309 forks source link

When submitting the body as a ZIO Stream, the response is received too early and with an error #2337

Open denisftw opened 6 days ago

denisftw commented 6 days ago

So, I'm trying to handle a pretty simple scenario. The client sends several strings

val stream = ZStream.fromIterable(List("1", "2", "4", "8", "9"))

separated by \n and encoded into bytes as a stream. The server is supposed to read them all and send back the count in the 200 OK response. However, it doesn't work.

Whenever I try to send a POST request like this

    val byteStream = ctns
      .via(ZPipeline.intersperse("\n") >>> ZPipeline.utf8Encode)
    val request = client3.basicRequest
      .post(countElementsUrl)
      .streamBody(ZioStreams)(byteStream)
      .response(client3.asStringAlways)

the response is returned before (!) it's completed on the server-side. The response is a 400 with empty body and a warning header that reads:

199 ZIO HTTP "text is empty (possibly HTTP/0.9)"

However, on the server, I can easily put a breakpoint and check all the elements of the original stream.

idea-debug

I created a repo with a minimal example: https://github.com/denisftw/sttp-zio-streaming-response It contains two main applications: HttpClientMain, which is a client that uses sttp, and HttpServerApplication, which is a server that uses ZIO HTTP.

Am I missing something here?

adamw commented 5 days ago

If the server responds with a 400, I guess it's a server problem, not a client one?

denisftw commented 5 days ago

Hi @adamw ! Thanks for a quick response!

I would think so, but even then, there are still two unsolved mysteries:

I investigated a bit more and noticed one thing that could probably explain why it doesn't work.

When I'm using Java 11's HttpClient as the backend, it adds tons of HTTP2-related headers:

headers-java11client

However, when I switch to AsyncHttpClient, there are only a few headers and nothing about HTTP2:

headers-asynchttpclient

In both cases I use the same code to form the request (shown above). Since AsyncHttpClient backend works and Java 11 doesn't, I assume that the problem is the headers or maybe inability of ZIO HTTP Server to work with it.

I also updated the sample repo, added an example with ZIO HTTP as a client (which also works and sends headers similar to AsyncHttpClient backend).

adamw commented 2 days ago

Maybe you can try writing a test using Java's HttpClient directly? If this shows incorrect behavior, then I think that would be a good bug report for ZIO HTTP (as it would demonstrate a failing test case).