search

grpc / grpc-swift

The Swift language implementation of gRPC.
Apache License 2.0
2.04k stars 420 forks source link

When GRPC backend server pod(Connection is terminated) is down, Not getting any callback. #1615

Closed shivamk-suki closed 1 year ago

shivamk-suki commented 1 year ago

Describe the bug

When GRPC backend connection is terminated or pod is down, we are not getting any callback so that we can try to re-connect. I have added the logger and got the below logs when the backend connection is terminated or pod is down-

2023-06-09T14:02:41+0530 trace suki-logger : h2_end_stream=true h2_payload=HEADERS grpc_connection_id=C1EBA4AA-83EF-4D2C-9904-391FF8C854BB/0 grpc_request_id=92E70F26-BC42-4918-8F52-8817FFCEB117 h2_headers=[([], ":status", "200"), ([], "content-type", "application/grpc"), ([], "grpc-status", "14"), ([], "grpc-message", "upstream connect error or disconnect/reset before headers. reset reason: connection termination"), ([], "date", "Fri, 09 Jun 2023 08:32:41 GMT"), ([], "server", "envoy")] received HTTP2 frame

Currently we have 2 delegate which is implemented(iOS-swift) in our code base- a) connectivityStateDidChange(from oldState: ConnectivityState, to newState: ConnectivityState) b) didCatchError(_ error: Error, logger: Logger, file: StaticString, line: Int)

But these are not getting called when above situation occurs. Just wanted to know how to handle above situation so that we can try to reconnect?

To reproduce

1 Connected with GRPC backend server. 2) From backend-> Connection is terminated(Pod is down). 3) Not getting any callback in this case.

Expected behaviour

We should get callback so that we can try re-connect.

Additional information

This is how we create a channel:-

Screenshot 2023-06-12 at 1 43 01 PM
glbrntt commented 1 year ago

From your logs it looks the connection from the envoy proxy to the upstream has been terminated; not the connection from the client to the proxy.

Generally speaking if a connection from the client is dropped, you should't need to do anything, gRPC will try to reconnect automatically.

araisuki commented 1 year ago

@glbrntt Say for example I am streaming the audio continuously using a stream RPC and if the server went down, How can I inform the user that the whatever the audio spoken are not reaching the server? I feel in this case the socket connection is closed between the client and server.

glbrntt commented 1 year ago

The RPC will fail with an error if the connection is dropped.