Circuit Breaking

[ignasi35]

Akka-gRPC client should allow setting up circuit breaking. Related to #137

[ignasi35]

Some mental notes:

A gRPC client has a Channel instance connecting the client to a single remote host and that Channel is shared across several calls.

Lagom's CircuitBreakingServiceLocator offers a different circuit breaker instance for each call on a remote service. Different approaches could be: (1) per remote service or (2) per remote instance. Maybe akka-grpc could consider supporting all 3 alternatives in the future.

Akka provides general purpose circuit breakers so users can manually setup circuit breaking around their GrpcService calls if they want to.

Lagom provides a general purpose circuit breaker panel and also promotes ServiceLocator implementors to extend CircuitBreakingServiceLocator which keeps circuit breaking a bit hidden but still exposed to users via Settings. CircuitBreakingServiceLocator looks up a service instance and then runs userland code wrapped on a circuit breaker.

[yahor-filipchyk]

gRPC project has this approved proposal for retries that includes retry throttling. I think it'd be nice if akka-grpc client could use the configuration and semantics written in the proposal, opening the door for future full support of gPRC standard, including service config discovery via the DNS or other service discovery mechanisms.

[raboof]

I agree this is an area where the Akka gRPC client can be improved! We should definitely look at the linked proposal, but also remember to look for the best ways to translate this into Akka ecosystem concepts (such as akka-discovery and eventually akka-http)

[ignasi35]

After digging a bit more on the topic, I've found a discussion hinting the grpc-java client can support circuit breaking via Interceptors (the discussion links to a PoC in Kotlin using resilience4J).

Using Interceptors is more convenient than my suggestion above because it sets up the circuit breaker per gRCP client automatically (instead of having to wrap every call on the client).

[ignasi35]

The implementation based on an Interceptor (in #1118) is not competing with the proposal but an alternative available in simpler deployments (where not DNS or similar config delivery mechanisms exist).

[ignasi35]

Hmm, maybe another possible approach would be to wrap the method invocations in the generated code?

The generated code fora client has access to, for example, the GrpcClientSettings.

[exp0nge]

Hey, any updates on this?

[He-Pin]

Hey, any updates on this?

Pr is welcome😃😃😃