Reconnect logic

[haf]

There have already been some questions about how to reconnect. My use-case is bi-directional streaming with gRPC; with a connection that is alive for as long as the app is alive (but like we know the OS might/will close connections after a while). I want to make sure the gRPC client reconnects and retries to send unsent messages.

These issues mentioned reconnection behaviour:

• https://github.com/grpc/grpc-dart/issues/180
• some improvements were made https://github.com/grpc/grpc-dart/pull/231
• and https://github.com/grpc/grpc-dart/issues/564 brushes off the question with:

  Right now the best solution you can come up with is to implement some code on the side to monitor network state changes and manually drop old connection and reconnect when network state changes.

I'd like to get an explainer on how exactly to write "some code".

For background, I keep track of the state like this:

      final sendStream = StreamController<AnalyseRequest>(
        onListen: () {
          loggy.debug('LISTEN ON request stream for listing$listingId');
        },
        onCancel: () {
          loggy.debug('CANCEL request stream for listing$listingId');
        },
        onPause: () {
          loggy.debug('PAUSE request stream for listing$listingId');
        },
        onResume: () {
          loggy.debug('RESUME request stream for listing$listingId');
        },
      );

      // kick-start the bi-directional streaming
      final responses = client.analyse(
        sendStream.stream,
        options: CallOptions(compression: const GzipCodec()),
      );

      // listen to the response stream / messages
      // ignore: cancel_subscriptions
      final subscription = responses.listen(
        _onAnalyseResponse(listingId),
        // ignore: avoid_types_on_closure_parameters
        onError: (Object error, StackTrace stackTrace) {
          loggy.error('request stream for listing$listingId error: $error', error, stackTrace);
          _events.sink.addError(error, stackTrace);
        },
      );

      _streams[listingId] = (responses, subscription, sendStream);

Then at some point, due to networks being networks; this gets logged:

flutter: 🐛 16:58:25.540942 DEBUG PROVIDER: StreamingAPIPredictor - CANCEL request stream for listingd55a919c-0e8b-419a-a1e0-a208e97a93b2 flutter: ‼️ 16:58:25.541688 ERROR PROVIDER: StreamingAPIPredictor - request stream for listingd55a919c-0e8b-419a-a1e0-a208e97a93b2 error: gRPC Error (code: 14, codeName: UNAVAILABLE, message: Missing trailers, details: null, rawResponse: null, trailers: {})

version: 3.2.4

The ultimate API for me would be that I can rely on the client to keep this stream open; how exactly this would work, I'm not sure, so I'm asking you.

Repro steps

1. set up a bidirectional stream
2. sleep the app
3. switch back to the app

OR

1. same
2. switch networks
3. try to send a request

Expected result: either that we can resume the connection (incl its state server-side with its python gRPC impl); or that there's some guidance on how to build resilient gRPC clients and hooks for me to hook into to resume.

Actual result: the app hangs, because the socket is dead and I don't see how to reconnect it

[VolodiaHunkalo]

@haf did you resolve this issue somehow? have similar problem with bi-directional gRPC stream

[haf]

No, no solution, still need one.

[VolodiaHunkalo]

as I correctly understand, you also need to know status of this stream to make either reconnect or close, yes? @haf

[VolodiaHunkalo]

it is inside http2 library, but it's hidden https://github.com/dart-lang/http2/blob/master/lib/src/streams/stream_handler.dart

[haf]

No I don't believe in statuses; if I try to send and get timeouts or ECONNRST or similar back, then I want to try to reconnect.

[VolodiaHunkalo]

how you handle this issue right now? with connectivity checker and forcefully close connection when change?

[haf]

I moved away from streaming for this reason; I don't have a solution yet. But if I really need one I'm going to dig into this library and maybe I'd wrap the channel or make it a factory method instead of an instance (if it needs to be recreated)