Add the ability to query addresses of replicas in the list of services

[davidfowl]

What should we add or change to make your life better?

Add the ability to get the addresses of replicas within the list of services.

Why is this important to you?

Some distributed algorithms require knowing what replicas exists so that pods/nodes can communicate between each other (membership algorithms etc).

This is also useful when trying to build out something like a client load balancer that wants to send traffic to individual replicas instead of being routed through the proxy.

[JamesNK]

I want this to support lookaside loading balancing in the gRPC client. I need to know the underlying addresses of replicas to call them directly.

Storing addresses in an environment variable would work for tye run at the replica IPs won't change, but Kubernetes pods and their addresses can change at runtime. Because we want a solution that works in both scenarios, relying completely on environment variables won't work.

In Kubernetes the replica IPs are resolved from DNS A records returned by Dns.GetHostAddressesAsync(host). Apps periodically poll for changes to get updates.

An idea for a consistent API:

public interface IAddressResolver
{
    Task<IList<string>> GetAddresses(string service);
}

internal class DefaultAddressResolver : IAddressResolver
{
    public DefaultAddressResolver(IConfiguration configuration) {}
    public Task<IList<string>> GetAddresses(string service) {}
}

The implementation use environment variables to determine whether it is running locally or in Kubernetes.

• If app is running locally then the replica addresses would be retrieved from an environment variable with a comma delimited value, e.g. SERVERNAME:REPLICA_ADDRESSES=127.0.0.2,127.0.0.3.
• If app is running in Kubernetes then the service name would be resolved to a Kubernetes service address, e.g. SERVERNAME.default.svc.cluster.local:8000 and Dns.GetHostAddressesAsync would be used to fetch the addresses

[JamesNK]

Related, how would you configure a headless service in tye.yaml? e.g. a Kubernetes service with clusterIP: None

[abbottdev]

I would like this to be able to also support Orleans services, where each Orleans silo needs to be able to communicate with another. There's an in-built clustering provider for Kubernetes, but this would only function once deployed.

[WoLfulus]

I would like this to be able to also support Orleans services, where each Orleans silo needs to be able to communicate with another. There's an in-built clustering provider for Kubernetes, but this would only function once deployed.

Same situation here.