d-exclaimation / over-layer

A GraphQL over Websocket Stream-based Transport Layer on Akka.
https://overlayer.netlify.app/
Apache License 2.0
2 stars 1 forks source link

Bug: GraphQL Subscription Source abruptly terminated after idle for longer than 30 seconds #8

Closed d-exclaimation closed 3 years ago

d-exclaimation commented 3 years ago

Description

I found an issue where the GraphQL subscription Akka Source died after 20-30 seconds of being idle (no data). Not sure the cause outside a timeout. Error thrown said Actor system or materializer was terminated when it's clearly not.

Callback to send a message to Ended message was not delivered, for unknown reason.

Need to fix. Either add a logic to reinitialize stream on failure or find out why the timeout happens in the first place.

d-exclaimation commented 3 years ago

Issue somewhat figured out. The stream has a timeout around 20-30 seconds of idle, but the Ended message was not delivered due to the websocket connection closing around the same time of being idle. Thus, resulting in a closed stream and a dead letter.

Resolving the issue by allowing stream to die after idle with now a specified timeout, and add logic to resubscribe on timeout failure.

Need to test scenario where resubscribe happened and message flow through websocket again making them not idle, and see whether the websocket and source stays alive.

d-exclaimation commented 3 years ago

Also need to update guide to warn of idle Source since the getting started example was using 1 singleton source.

d-exclaimation commented 3 years ago

I am also considering other reactive stream options to see where they stack up. If using a different streaming option produces the same problem, it might just be either sangria executor or how I timeout the subscription.

Using Akka stream allow me to save dependencies by using what's already a dependency but if using an alternative prove to be a better option, I think it's best to go that route.

d-exclaimation commented 3 years ago

Mostly solved with these two commits

and with the latest release of SubPub