IMemberDiscoveryService

[LCastilloSymbiotic]

Hi,

Do you have a example to use IMemberDiscoveryService? I would like to add members dynamically in the cluster...

Best regards,

Luis

[sakno]

In 3.x, there is no example for IMemberDiscoveryService because its actual implementation fully depends on the external discovery mechanism. It can be Consul, etcd, Kubernetes or even file watcher. However, you must use IMemberDiscoveryService carefully: one node can be removed or added at a time. This restriction is not implemented by the interface or underlying Raft implementation. Otherwise, there is a non-zero probability to have more than one leader in the cluster for a short period of time.

In 4.x, this situation is solved completely. No more IMemberDiscoveryService or configurable list of cluster members. Raft defines its own membership protocol that is implemented in the upcoming major version. For now, 4.x is available as a preview package on NuGet.

[LCastilloSymbiotic]

Ok ok, a question in version 4.x are there examples of how to use this new union members model?

Best regards,

Luis

[sakno]

In case of 4.x, the programming model follows recommendations from Raft paper:

1. Get IRaftHttpCluster interface from DI
2. Call AddMemberAsync or RemoveMemberAsync
3. Optionally register an instance of ClusterMemberAnnouncer in DI

Also I recommend to read this article about node bootstrapping.

[LCastilloSymbiotic]

Hi In 3.x, Is it possible to add a node manually?

I am trying something like this:

private readonly IRaftCluster _cluster;

IPEndPoint checkIP = new IPEndPoint(IPAddress.Loopback), 3262);
_cluster.Members.Aggregate(checkIP);

But I get an error for the type as it requests an IClusterMember

[sakno]

Yes, the configuration is tracked by RaftCluster at runtime via IOptionsMonitor. So you can change the configuration at run-time. If you're using the file for storing configuration, then just open and edit it (add new member to members section).

[LCastilloSymbiotic]

Don't you have an example of how to do that?

This is my normal host creation:

 private static Task UseAspNetCoreHost(int port, string? persistentStorage = null)
        {
            var configuration = new Dictionary<string, string>
            {
                {"partitioning", "false"},
                {"lowerElectionTimeout", "150" },
                {"upperElectionTimeout", "300" },
                {"requestJournal:memoryLimit", "5" },
                {"requestJournal:expiration", "00:01:00" }
            };

            //Set local URI
            LocalURI = String.Format("{0}{1}", "https://", NodesMembers[0].ToString());

            //Set nodes members
            for (int i = 0; i < NodesMembers.Count; i++)
            {
                configuration.Add(String.Format("members:{0}", i),
                    String.Format("{0}{1}", "https://", NodesMembers[i].ToString()));
            }

            if (!string.IsNullOrEmpty(persistentStorage))
                configuration[CustomPersistentState.LogLocation] = persistentStorage;

            return new HostBuilder().ConfigureWebHost(webHost =>
            {
                webHost.UseKestrel(options =>
                {
                    options.ListenLocalhost(port, listener => listener.UseHttps(LoadCertificate()));
                })
                .UseStartup<StartupReplication>();

            })
            .ConfigureLogging(builder => builder.AddConsole().SetMinimumLevel(LogLevel.Error))
            .ConfigureAppConfiguration(builder => builder.AddInMemoryCollection(configuration))
            .JoinCluster()
            .Build()
            .RunAsync();
        }

[sakno]

Raft impl for ASP.NET Core fully relies on Configuration Model and Options pattern from the framework:

• Configuration
• Options

As far as I know, in-memory configuration doesn't support change tracking. Therefore, you need to write your own configuration provider with in-memory storage capabilities or change another source of configuration, e.g. json file. You can find a lot of examples in the articles mentioned above.

[LCastilloSymbiotic]

Writing configs in eg .json doesn't make it static? I think about it in this sense, I imagine that the data from the configuration file is loaded when the cluster is created, but once it is up, it does not check the file again or does it?

[sakno]

It does.

[LCastilloSymbiotic]

When I insert the configuration from a dictionary like this:

var configuration = new Dictionary<string, string>
            {
                {"partitioning", "false"},
                {"lowerElectionTimeout", "150" },
                {"upperElectionTimeout", "300" },
                {"requestJournal:memoryLimit", "5" },
                {"requestJournal:expiration", "00:01:00" },
                {"members:0", "https://192.168.31.144:3262" }
            };

As I understand it, the equivalent in JSON is the following file:

{
  "partitioning": "false",
  "lowerElectionTimeout": "150",
  "upperElectionTimeout": "300",
  "requestJournal": {
    "memoryLimit": "5",
    "expiration": "00:01:00"
  },
  "members": {
    "0": "https://192.168.31.144:3262"
  }

}

When I insert the dictionary configuration like this, everything works perfectly:

.ConfigureAppConfiguration(builder => builder.AddInMemoryCollection(configuration))

But if I insert it via JSON this way it doesn't work, do you know why?

.ConfigureAppConfiguration(builder => builder.AddJsonFile("settingsRaft.json", optional: false, reloadOnChange: true))

[LCastilloSymbiotic]

If you had an example I would really appreciate it.

[sakno]

I'm not sure that the configuration mentioned above satisfies the requirements of JSON-based configuration in ASP.NET Core. Probably, the array or members should be defined in another way.

[sakno]

.NET JSON configuration uses JSON arrays instead of indexing array elements directly:

{
  "members": ["https://192.168.31.144:3262"]
}

[sakno]

Closing the issue due to inactivity.