UseLocalHostClustering with two .NET Core Web API hosted silos

[sjbthfc2]

Hi,

I know a similar issue has been raised that suggested it is possible to have multiple silos when using UseLocalHostClustering, however, I cannot seem to get this to work with two .NET Core 3.2 WebApi hosts. I am creating a silo and client in the host configuration of each of the APIs and in the client config specifying the range of ports that both of the silos expose.

When I try and connect the clients to the silos they hang on the Connect().Wait() call. Everything works fine if I just use the client created by default and do not try and configure my own one. What am I doing wrong and why is the silo not available for the client to connect to.

Many thanks.

`using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Hosting; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Hosting; using Orleans; using Orleans.Configuration; using Orleans.Hosting; using OrleansPoc.GrainInterfaces; using System.Net; using System.Threading.Tasks;

namespace OrleansPoc.SiloOne { public class Program { public static Task Main(string[] args) => Host.CreateDefaultBuilder(args) .ConfigureWebHostDefaults(webBuilder => { webBuilder.Configure((ctx, app) => { if (ctx.HostingEnvironment.IsDevelopment()) { app.UseDeveloperExceptionPage(); }

                           app.UseHttpsRedirection();
                           app.UseRouting();
                           app.UseAuthorization();
                           app.UseEndpoints(endpoints =>
                           {
                               endpoints.MapControllers();
                           });
                       });
                   })
                    .UseOrleans(siloBuilder =>
                    {
                        siloBuilder
                        .UseLocalhostClustering()
                        .Configure<ClusterOptions>(opts =>
                        {
                            opts.ClusterId = "dev";
                            opts.ServiceId = "OrleansPocIotService";
                        })
                        .Configure<EndpointOptions>(opts =>
                        {
                            opts.AdvertisedIPAddress = IPAddress.Loopback;
                        });
                    })
                   .ConfigureServices(services =>
                   {
                       services.AddControllers();
                       services.AddSingleton<IClusterClient>((s) =>
                       {
                           var client = new ClientBuilder()
                               // Clustering information
                               .Configure<ClusterOptions>(options =>
                               {
                                   options.ClusterId = "dev";
                                   options.ServiceId = "OrleansPocIotService";
                               })
                               // Clustering provider
                               .UseLocalhostClustering(new int[] { 30000, 29999 })
                               // Application parts: just reference one of the grain interfaces that we use
                               .ConfigureApplicationParts(parts => parts.AddApplicationPart(typeof(ISystemGrain).Assembly))
                               .Build();

                           client.Connect().Wait();
                           return client;
                       });
                   })
               .RunConsoleAsync();
}

} `

[JorgeCandeias]

Look at the health check sample's Program.cs for a way to do this.

When running multiple silos in one machine, those silos must have different ports. In addition, when using the local host clustering provider, you must have one silo acting as service discovery. This is a well-known silo that other silos will connect to in order to form a cluster with. This does not apply when using an external cluster membership provider.

In this code from the sample:

builder.UseLocalhostClustering(siloPort, gatewayPort, new IPEndPoint(IPAddress.Loopback, 11111));

the siloPort is set as the first available port starting from 11111 at the time you're starting the particular silo instance. This makes the first silo you spin up take that port and the following silos take ports 11112, 11113, etc. The new IPEndPoint(IPAddress.Loopback, 11111) argument tells the following silos to connect to the first one you started in order to join the cluster.

[sjbthfc2]

Hi @JorgeCandeias, I have the loopback endpoint, but I still get client timeout when it tries to connect to a silo. I have decided to try using AzureStorageClustering, but again, the silos are not registered in time before the client attempts to connect so it cannot find any gateway.

I am struggling with how to best host a silo and a client in an ASP.NET Core 3.0 Web API. Is it possible, I have seen it mentioned but not sure how to do it? Do silos have to be completely seperate host processes from the client?

Thanks

[JorgeCandeias]

Assuming you only want to connect to a single Orleans cluster (as your code suggests) then you don't need the client at all in a co-hosting setup. The IGrainFactory service is provided for free and you can request it via DI from any controller. There is no need to configure a client at all of wait for it to connect. You can remove that code altogether.

Regarding registration in the host builder, it is best to tack on .UseOrleans() before tacking on ConfigureWebHostDefaults() and other web-api related services. This is because registration order in the host builder also affects in what order the underlying hosted services are started, and, to minimize noise and failed api requests during startup, we want the Orleans relevant services to spin up and stabilize before the we allow the web ones to do so or our own application ones if we have them.

Below is some code I've copied from a new sample I'm working on. It just happens to spin up an Orleans silo and Web Api (including the Swagger UI) in the same process, for ease of testing the subject of the sample. This project is using the web sdk, same as a typical asp.net core project created from visual studio. Note that it's still missing the siloPort hack for development, but you can add that on top.

namespace Application
{
    public static class Program
    {
        public static async Task Main(string[] args) =>
            await CreateHostBuilder(args).Build().RunAsync().ConfigureAwait(false);

        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .ConfigureLogging(logging =>
                {
                    logging
                        .ClearProviders()
                        .AddSerilog(new LoggerConfiguration()
                            .WriteTo.Console()
                            .CreateLogger(), true);
                })
                .UseOrleans(silo =>
                {
                    silo
                        .ConfigureApplicationParts(apm => apm.AddApplicationPart(typeof(SimpleThreadPoolWorkGrain).Assembly).WithReferences())
                        .UseLocalhostClustering()
                        .UseDashboard(options =>
                        {
                            options.Port = 5002;
                        });
                })
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.ConfigureServices(services =>
                    {
                        services.AddControllers();
                        services.AddSwaggerGen(options =>
                        {
                            options.SwaggerDoc("v1", new OpenApiInfo
                            {
                                Title = nameof(Application),
                                Version = "v1"
                            });
                            options.IncludeXmlComments(Path.Combine(AppContext.BaseDirectory, $"{typeof(Program).Assembly.GetName().Name}.xml"));
                            options.EnableAnnotations();
                            options.DescribeAllParametersInCamelCase();
                        });
                    });

                    webBuilder.Configure((context, app) =>
                    {
                        if (context.HostingEnvironment.IsDevelopment())
                        {
                            app.UseDeveloperExceptionPage();
                        }

                        app.UseHttpsRedirection();
                        app.UseRouting();
                        app.UseAuthorization();

                        app.UseSwagger();
                        app.UseSwaggerUI(options =>
                        {
                            options.SwaggerEndpoint("/swagger/v1/swagger.json", $"{nameof(Application)} v1");
                        });

                        app.UseEndpoints(endpoints =>
                        {
                            endpoints.MapControllers();
                        });
                    });
                })
            .ConfigureServices((context, services) =>
            {
                services.Configure<SampleOptions>(options => context.Configuration.GetSection("Sample").Bind(options));
            });
    }
}

[sjbthfc2]

Hi, that is working perfectly now with two silos in WebAPI hosts using Azure Table cluster storage. I didn't think to just use GrainFactory when the silo is being accessed in the same process.

Thank you very much for your excellent example and help.