Open maximskripnik opened 6 months ago
maximskripnik
commented
6 months ago It seems like the issue is with zio-http, caused by this: https://github.com/zio/zio-http/issues/2584
It's already fixed with https://github.com/zio/zio-http/pull/2599, so I guess we just need to wait for the next zio-http release and update it in tapir
TrustNoOne
commented
5 months ago zio-http was released https://github.com/zio/zio-http/releases/tag/v3.0.0-RC5
adamw
commented
5 months ago @TrustNoOne yes, but it cointains snapshot transitive dependencies, so we cannot use it
adamw
commented
5 months ago 
dacr
commented
4 months ago I've just checked, https://github.com/zio/zio-http/issues/2584 hang issues I had previously and coming from the ZIO side, are fixed (zio 2.0.22, tapir 1.10.5, zhttp-3.0.0-RC6), but now I have something wrong when tapir try to process a stream which returns an error.
I get 200 OK with no content :
< HTTP/1.1 200 OK
< Content-Type: application/json-seq
< transfer-encoding: chunked
<
* transfer closed with outstanding read data remaining
* Closing connection 0instead of getting (with the same inputs) :
< HTTP/1.1 400 Bad Request
< content-length: 305
< Content-Type: application/json
<
* Connection #0 to host 127.0.0.1 left intact
{"querySyntaxError"...I had two similar endpoints with the same entries, the first one returns a stream and the second one the result of a backend consumed stream (the same one).
If needed I will be able to write some snippets to reproduce the issue but not before 3 weeks unfortunately.
adamw
commented
4 months ago @dacr please do, I'm not aware of any similar issues
dacr
commented
3 months ago Consider the stream logic failingGreetingStream3 bound to helloEndPoint, it summaries the issue I have, and once written like this it also gives me some hints on what's going on. I try to build a ZIO Stream from a java stream which can fail, and while reading this I realized the issue is coming from my side and the way I'm building the stream :(
val failingGreetingStream3: ZIO[Any, String, ZStream[Any, Throwable, Byte]] = ZIO.succeed(
ZStream.fromJavaStreamZIO(
ZIO.fail(Exception("Can not build the stream"))
)
)
val helloEndPoint =
endpoint
.description("Returns greeting")
.get
.in("hello")
.out(streamBody(ZioStreams)(Schema.derived[Greeting], JsonSeqCodecFormat()))
.out(statusCode(StatusCode.Ok).description("query success"))
.errorOutVariantPrepend(oneOfVariant(StatusCode.InternalServerError, plainBody[String]))
val helloRoute = helloEndPoint.zServerLogic[Any](_ => failingGreetingStream3)The full script :
// ---------------------
//> using scala "3.4.1"
//> using dep "com.softwaremill.sttp.tapir::tapir-zio:1.10.7"
//> using dep "com.softwaremill.sttp.tapir::tapir-zio-http-server:1.10.7"
//> using dep "com.softwaremill.sttp.tapir::tapir-json-zio:1.10.7"
// ---------------------
import sttp.tapir.ztapir.*
import sttp.tapir.server.ziohttp.ZioHttpInterpreter
import zio.*
import zio.stream.*
import zio.json.*
import zio.http.Server
import sttp.capabilities.zio.ZioStreams
import sttp.model.{MediaType, StatusCode}
import sttp.tapir.{CodecFormat, Schema}
import sttp.tapir.generic.auto.*
import sttp.tapir.json.zio.*
import sttp.tapir.ztapir.*
case class Greeting(message:String) derives JsonCodec
case class JsonSeqCodecFormat() extends CodecFormat {
override val mediaType: MediaType = MediaType.unsafeApply("application", "json-seq")
}
object WebApp extends ZIOAppDefault {
// --------------------------------------------------
val greetingStream: ZIO[Any, String, ZStream[Any, Throwable, Byte]] = ZIO.succeed(
ZStream
.repeat(Greeting("Hello world"))
.schedule(Schedule.spaced(1.second))
.flatMap(greeting => ZStream.fromIterable( (greeting.toJson+"\n").getBytes ))
)
val failingGreetingStream1: ZIO[Any, String, ZStream[Any, Throwable, Byte]] = ZIO.fail(
"Can not build the stream"
)
val failingGreetingStream2: ZIO[Any, String, ZStream[Any, Throwable, Byte]] = ZIO.succeed(
ZStream.fail(Exception("Can not build the stream"))
)
val failingGreetingStream3: ZIO[Any, String, ZStream[Any, Throwable, Byte]] = ZIO.succeed(
ZStream.fromJavaStreamZIO(
ZIO.fail(Exception("Can not build the stream"))
)
)
val helloEndPoint =
endpoint
.description("Returns greeting")
.get
.in("hello")
.out(streamBody(ZioStreams)(Schema.derived[Greeting], JsonSeqCodecFormat()))
.out(statusCode(StatusCode.Ok).description("query success"))
.errorOutVariantPrepend(oneOfVariant(StatusCode.InternalServerError, plainBody[String]))
//val helloRoute = helloEndPoint.zServerLogic[Any](_ => greetingStream)
val helloRoute = helloEndPoint.zServerLogic[Any](_ => failingGreetingStream3)
val routes = ZioHttpInterpreter().toHttp(List(helloRoute))
override def run = Server.serve(routes).provide(Server.default)
}
WebApp.main(Array.empty)@dacr does this work when using zio-http directly, without tapir?
Tapir version: 1.9.10
Scala version: 3.3.1
Connection hangs with no useful logs when response zio stream fails
When using zio streams to serve http response using
ZioHttpInterpreter, the server does not properly handle stream failures. The client that sent request hangs, so apparently the server doesn't even close the connection. On top of that, server logs are full of Netty internal exceptions instead of the root cause exception that failed the stream. Additionally, elements from the stream preceding the failure are not sent in the response as wellHow to reproduce?
Server code
This example spins up a server with one endpoint
/testthat returns a simple plain text stream that fails after two elements are emittedRunning the server
Sending the request
You can observe two issues here:
Server logs
Meanwhile the server just spits this large pile of Netty internals. The big issue here is the lack of information about the throwable that failed the response stream (no 'boom' string seen anywhere in the logs)
Additional information
Expected behavior
The expected behavior in this case is that the server:
Working example
As a side note, this example shows that such behavior did exist in tapir v1.2.3
The logs above contain the information about the root cause exception, even though there is also not a very useful warning from Netty