Closed jdom closed 6 years ago
Lots of stuff here that I need to read, but a few things to make note of that I think will impact it while I get to reading.
We are in the process of generalizing some of the hosting concepts that I think will help with this. It falls into a few main features:
IHostedService
and the non-web specific part of IHostingEnvironment
into an abstractions package that is separated from Hosting.HostBuilder
and Host
that are distinct from WebHostBuilder
and don't have as many of the web/ASP.NET specific stuff.What that will give you is a HostBuilder
that lets you setup Configuration, Logging, and DI the same was as ASP.NET Core, but not have any of the rest of the ASP.NET Core specific stuff. It will also not have Startup.cs, it would only allow configuring in Program.cs but there are some changes happening to the surface area of that to make that a bit more natural.
The IHost
built by this HostBuilder
would control the lifecycle of any IHostedServices
that have been registered in DI (https://github.com/aspnet/Hosting/blob/a8c61b5abccfbce28f1aa21dd967462c7d3bf0ca/src/Microsoft.AspNetCore.Hosting.Abstractions/IHostedService.cs) A HostedService is just a type that has a Start
and Stop
method.
We are doing this so that folks can do exactly the sort of stuff that you are describing here. So we should talk more about it :).
A rough code example would be something like this:
var host = new WebHostBuilder()
.UseContentRoot(Directory.GetCurrentDirectory())
.ConfigureAppConfiguration((context, builder) => builder
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{context.HostingEnvironment.EnvironmentName}.json", optional: true)
.AddEnvironmentVariables())
.ConfigureLogging(logger => logger
.AddConsole()
.AddDebug())
.ConfigureServices(services => services
.AddMyServices()
.AddSomeOtherService())
.UseSomeHostedService()
.Build();
host.Run();
In this example the UseSomeHostedService method would add an IHostedService
to DI, which means it could also be done in ConfigureServices
if that made more sense.
Anyway, I need to read more of your issue but I wanted to get this information here for you to consider sooner rather than later.
That is great news @glennc, thanks for the feedback. 1) Is there a tentative release/timeframe that you are targeting? 2) What's the reason to not generalize the Startup convention too? Seems really useful to split host from application configuration. 3) After you finish reading, I'm particularly interested in feedback for the configuring named services (not just adding multiple hosted services of different types), as I couldn't find any parity in ASP.NET Core, but I'm certain that I'm missing something, as it's not a contrived feature.
I like that in this new generalized abstraction you can explicitly configure the configuration builder that's used by the host.
BTW, I'm aware of IHostedService
, and in my prototype I actually used it for some of the services. Nevertheless we still need to iterate a little bit more on it, since starting up an Orleans silo has a much longer lifecycle, where some services need to start at different stages. This might be solved by not using IApplicationLifecycle
as-is, but have a more flexible lifecycle implementation.
An alternative to your separate Configure
method might be something that's also done by e.g. MVC and IdentityServer4, where you can configure these modules via a separate IMvcBuilder or IIdentityServerBuilder directly in ConfigureServices
. This would allow you to add/change services to the DI within your Orleans configuration.
public void ConfigureServices(IServiceCollection services)
{
services.AddIdentityServer()
.AddTemporarySigningCredential()
.AddInMemoryPersistedGrants()
.AddInMemoryIdentityResources(Config.GetIdentityResources())
.AddInMemoryApiResources(Config.GetApiResources())
.AddInMemoryClients(Config.GetClients())
.AddAspNetIdentity<ApplicationUser>();
}
Thanks @cwe1ss, I'll continue to look, but I'm initially not seeing how those 2 builders would help with configuring different named instances, since they both seem to be just configuring the wrapped IServiceCollection
. So in theory if I had 2 different AzureBlobStorageProvider instances I want to configure and they both take different AzureBlobStorageOptions
, they would end up configuring the single instance of the options in the service collection.
I take it that having an IMvcBuilder
and related helps with the configuration API discoverability, which is good, but doesn't tackle named service instances. Or were you suggesting this not as an alternative to configuring named services, but of something else?
The other thing I could take from your comment is that you might have been suggesting not to pass a delegate for configuring the named instance object, but instead return that builder. In that case, that could look something like this:
public void Configure(ISiloBuilder app)
{
app.ConfigureStorageProviders()
.AddMemory("Default")
.AddAzureBlob("AzureBlobUsingFluentConfig")
.Configure(Configuration.GetSection("StorageProviders:AzureBlob1"))
.Configure(options => options.ContainerName = "overriden");
// or even here potentially a different builder returned by each provider type, so that's it's more straightforward to configure
}
I actually implemented this in my prototype, but didn't put it in this design thread, as I thought it less readable, but if that's more aligned with ASP.NET, I would definitely consider so (TBH, I didn't polish that approach much, so it could be made more readable if that's the desired way to go). The
We are in the process of generalizing some of the hosting concepts that I think will help with this.
@glennc, Great! My hope is that we can adapt Orleans Virtual Actor model to run on top of a service framework abstraction so we can focus more on our core value add, the virtual actor model itself, without having to maintain an entire service framework just to run the model on top of.
Asp's hosting is close to what we need, but is currently too specific to web. As Julian's "De-inventing the wheel" blog post points out, we're looking more to de-invent at this point, so if you all are already working on this, I'd much prefer we contribute to it's success than duplicate your effort.
@jdom How about something like this? The advantage would be that adding services and configuration is in one place.
// Configuration usage
public void ConfigureServices(IServiceCollection services)
{
services.AddOrleans()
// Set global orleans options (you could also add nicer extension methods for this)
.Configure(options =>
{
options.SomeGlobalFlag = true;
})
// Extension methods for adding storage providers
.AddMemoryStorageProvider("Default")
.AddAzureBlobStorageProvider("AzureBlob", options => ...)
// Similar things would exist for stream providers
.AddAzureEventHubStreamProvider("name", options => ...);
}
// If you want more fine-grained options classes instead of the one big OrleansOptions-class
// you could certainly do that as well.
public class OrleansOptions
{
// This will be set by the ConfigureOrleansOptions class below
public IServiceProvider ApplicationServices { get; set;}
public bool SomeGlobalFlag { get; set; }
public Dictionary<string, IStorageProvider> StorageProviders { get; set; }
public Dictionary<string, IStreamProvider> StreamProviders { get; set; }
}
// This will populate the service provider when OrleansOptions-instance is created.
public class ConfigureOrleansOptions : IConfigureOptions<OrleansOptions>
{
private readonly _applicationServices;
public ConfigureOrleansOptions(IServiceProvider applicationServices)
{
_applicationServices = applicationServices;
}
public void Configure(OrleansOptions options)
{
options.ApplicationServices = _applicationServices;
}
}
public interface IOrleansBuilder
{
public IServiceCollection Services { get; }
}
public class OrleansBuilder : IOrleansBuilder
{
public IServiceCollection Services { get; }
public OrleansBuilder(IServiceCollection services)
{
Services = services;
}
}
public static class OrleansConfigurationExtensions
{
public static IOrleansBuilder AddOrleans(this IServiceCollection services)
{
var orleansBuilder = new OrleansBuilder(services);
// Add global services
orleansBuilder.Services.AddTransient<IConfigureOptions<OrleansOptions>, ConfigureOrleansOptions>();
orleansBuilder.AddCoreStorageProviderServices();
orleansBuilder.AddCoreStreamProviderServices();
return orleansBuilder;
}
// All Orleans-specific extensions hook onto IOrleansBuilder for nice IntelliSense
public static IOrleansBuilder Configure(this IOrleansBuilder builder, Action<OrleansOptions> options)
{
builder.Services.Configure<OrleansOptions>(options);
return builder;
}
public static IOrleansBuilder AddMemoryStorageProvider(this IOrleansBuilder orleansBuilder, string name, Action<MemoryStorageProviderOptions> providerOptions = null)
{
// Add provider-specific services to DI (if they don't yet exist)
orleansBuilder.Services.TryAddTransient<MemoryStorageProvider>();
orleansBuilder.Services.TryAddTransient<IDependencyOfMemoryStorageProvider, SomeDependencyOfMemoryStorageProvider>();
// Create the provider instance and add it to the options
orleansBuilder.Configure(o =>
{
// Create provider options instance from delegate
MemoryStorageProviderOptions providerOptionsInstance = new MemoryStorageProviderOptions();
providerOptions?.Invoke(providerOptionsInstance);
// Create the provider by using the provider specific options
// (ActivatorUtilities is from "Microsoft.Extensions.DependencyInjection")
MemoryStorageProvider provider = ActivatorUtilities.CreateInstance<MemoryStorageProvider>(o.ApplicationServices, providerOptions);
o.StorageProviders.Add(name, provider);
});
return builder;
}
}
// Getting all storage providers
public class OrleansStorageProviderManager
{
private readonly Dictionary<string, IStorageProvider> _storageProviders;
public SomeOrleansService(IOptions<OrleansOptions> options)
{
_storageProviders = options.Value.StorageProviders;
}
public IStorageProvider GetStorageProvider(string name)
{
return _storageProviders[name];
}
}
PS: this code was written in notepad so some stuff might be wrong.
Seems like aspnet/Options will get some kind of support for named options as well - I haven't looked into that yet though.
I think the main point of named services is the ability to resolve it by a string
name from DI container. Something similar to Named Services from Autofac and other containers. That enable what @jdom suggested to have multiple storage providers implementations. The idea is to have it on DI and not keep an internal collection I guess...
The named service issue, imo, is not really related to the builder or a general hosting framework. It's related to the DI abstraction. It's come up as part of the hosting problem because our current custom infrastructure provides this for us so if we replace it with something more generic, then we'd need to solve this somehow.
My hope was that future versions of the DI abstraction would address this. Our wish to use DI for this is mainly so we can take advantage of the scoping support containers provide, and also, in part, due to the expectation that DI 'should' solve this because other containers (AutoFac) have this capability.
Having said all of that, our current logic that handles this problem does not provide scoping capabilities, so for simple feature parity, this is not hard to solve. It's only hard because we're trying to solve it in DI to get scoping.
In short, imo, this is a di problem not a generic framework problem, and resolving it is not necessary for the generic hosting framework to provide us feature parity with what we currently have.
I agree @jason-bragg.
My point is, according to @cwe1ss' suggestion, we would have a dictionary inside a OrleansStorageProviderManager
to get the named provider.
What I meant, and you agree with, is that it must be resolvable thru DI and not that we have some collection to hold it. Like you said, the scope and naming of a dependency is something already sorted by most of the DI containers (i.e. AutoFac, Unity, etc.).
yep - my suggestion would have application-wide instances. If you need named scoped services, you could store the metadata (name, type, providerOptions) in a global service (instead of the actual instances) and get the providers resolved within your scope.
ASP.NET MVC does something similar with their action filters:
See MvcOptions (stores the global FilterCollection), FilterCollection (contains the metadata about filters), ServiceFilterAttribute (creates instances of the filter using ActivatorUtilities).
But obviously, you would have to do this yourself as well. It's not too complicated though IMO and I guess the performance wouldn't be much worse than using a built-in feature of another container since it's a very small additional layer.
PS: I agree that it would be nice to have named services in Microsoft.Extensions.DependencyInjection for your scenario, but I doubt that they will add it anytime soon because then some 3rd party containers probably can no longer comply.
PPS: I don't know anything about Orleans. If my input is not useful/helpful don't hesitate to tell me. :smile:
@cwe1ss no! Please, we appreciate your input :)
@cwe1ss this is very valuable, thanks. In fact, I was not aware of IConfigureNamedOptions. I had a similar design in an early prototype, but felt like I was re-inventing the entire Options package just to have that extra feature, so I dismissed that approach early. But it's good to know that something is coming to address a similar need to ours.
We just posted an update about this https://github.com/aspnet/Hosting/issues/1163
@jdom Should we close this one now?
I would really like to have a solution to use an existing service collection. A simple services.AddOrleans()
would be awesome :)
We implemented a lot of this, and also a lot changed its shape to align with the generic HostBuilder
from Microsoft.Extensions.Hosting
(still not released).
All extension methods currently support ISiloHostBuilder and IServiceCollection, so it should be possible I think (unless of course we have bugs)
Although there is nothing like just AddOrleans()
with defaults and everything just works.
The plan is that when the generic HostBuilder
is released, Orleans should be able to be hosted in that one (and of course allow co-hosting with ASP.NET Core and any framework that adds support for the generic host builder.
So you would implement a IHostLifetime, e.g. OrleansHostLifetime? And then I can have for example an AspNetHostLifetime, GrpcHostLifetime and OrleansHostLifetime in the same application?
@SebastianStehle almost: we would implement IHostedService
. IHostLifetime
is for controlling the lifecycle of the application itself. Eg to map it to a Windows Service, Cloud Service, Console Application (Ctrl+C support / signal handling) etc.
@ReubenBond Hello.
Sorry for asking in the closed issue but could not find any additional information regarding the current situation with Orleans
and GenericHost
. Are there any issue created for this - whether this is supported or someone is working on this? Interested in Console Application + Orleans.
Should it at the moment be created manually?
The generic host was released already. Nevertheless Orleans still doesn't support it natively.
You can of course implement the integration manually, nevertheless most extension methods extend the OrleansHostBuilder
and not IServiceCollection
, so it might be a little bit challenging to support the generic HostBuilder
externally.
The work to do to support the generic host is not that much, so if you are willing, perhaps you could send a PR instead of maintaining your own version? There are a few design questions to answer (especially with regards to whether there is a sub-builder hanging from the generic host builder, and with regards to back compat), but if you are willing, we could answer those.
@ReubenBond is currently out for a couple of weeks.
I forgot to mention that this OrleansHostBuilder
we built is very similar to the generic host builder, it's just that we released earlier, but we still made it familiar so that migrating to the once-released generic host would be straightforward.
@jdom Thanks a lot for the information. Understood.
I'm just started working with Orleans
and will play with it anyway. Once I get more familiar with the system and get a working version I'll come back with the options that I have.
Thanks.
What we want to do
Imitate ASP.NET Core's configuration and startup system, which is the evolution of very conscious improvements over several years. By standing on their shoulders we:
In its simplest form, an Orleans silo should be configured in a few lines of code, but allow more extensive configuration tweaking via code, or explicitly pulling in bits of declarative configuration (from XML, json, cfcfg, etc) via the built-in Options mechanism in ASP.NET Core and most .NET Standard hostable libraries (Microsoft.Extensions.Configuration and Microsoft.Extensions.Options are nuget packages not tied to ASP.NET itself)
The intention is to stop using the existing
ClusterConfiguration
object that is geared mainly towards declarative configuration in XML, and hence constraint a lot of what we can achieve with static configuration, as opposed to modifying the services collection directly and leveraging DI for configuration and using strongly typed objects or references to other live services.ASP.NET Core introduction
These are some good resources to understand ASP.NET Core's approach, and will be extremely valuable to understanding the new design (especially the first 2 resources):
Applying it to Orleans
The most important aspect of the new configuration system is the
Startup
class that must be defined by the end user. In its most basic form, the user will need to provide aConfigureServices
method that configure certain aspects of it, and also aConfigure
method that runs after the DI container is initialized but before the Silo starts.There is a lot of flexibility that the
Startup
class provides, such as using the environment name to optionally call different methods when configuring the services, or doing method injection on the Configure method. You can read more of these conventions in Application Startup.Some aspects worth highlighting for people not familiar with this approach in ASP.NET:
AddAzureTableMembership
would only appear when referencing the Azure integration package for Orleans.Initialize
method that gets invoked in the implementation with a configuration object that is essentially a string property bag.ConfigureServices
method). This opens up the door for huge opportunities.Named services (providers)
Where possible, we would attempt to align to the configuration approach used in ASP.NET Core, EntityFramework Core and existing library implementations that follow this model. There are a few cases where we have some existing features that deviate a little from their usage, such as configuring named services (providers) that could potentially even be of the same type (ie: there might be two Azure Blob Storage providers configured with different settings). Most of ASP.NET deals with configuring THE single service that implement a certain service interface. For these, I have a proposal that looks like the following:
There's a few aspects worth highlighting since there is no precedent in ASP.NET Core for named services configuration:
Configure
method as opposed toConfigureServices
) as these do not configure the container itself, but add new provider instances to a certain provider manager..Configure
extension methods that can take either declarative configuration or a delegate to tweak configuration. Nevertheless these options are not globally accessible from the container, and instead are provided only to the named services being built. In contrast, callingservices.Configure<AzureBlobStorageOptions>(options => options.ContainerName = "MyApp")
on the service collection itself as is typical in ASP.NET would mean thatIOptions<AzureBlobStorageOptions>
can be injected in any object from DI and would get the configured options. As it's obvious by now, if we did that it would mean that developers wouldn't be able to provide 2 distinct options when configuring 2 different Azure Blob storage providers.Logging abstractions
As proposed in Issue #2814: Use Microsoft.Extensions.Logging for logging abstractions, we want to leverage this abstraction that is used mainly by ASP.NET Core (but others are picking it up too) so that we get out of the logging abstractions business. Using the existing
LogManager
that we currently have would require a few changes to make it non-static and also make it DI friendly for allowing different consumers that leverage the more flexible configuration system. Doing those upgrades would probably be costly, and also not align with our vision of externalizing what makes sense. Also, leveraging the Microsoft.Extensions.Logging abstraction means that every 3rd party integration built for it by users of ASP.NET will automatically work with Orleans. We can still provide a small adapter to hook up existing customILogConsumer
implementations to it, to provide some easier backwards compatibility support. Design of this migration is outside of the scope of the Silo Builder, nevertheless it's a prerequisite to make the system as flexible as it intends.Hosting
There will be different ways to integrate with different hosts. For example, the whole
AzureSilo
façade that we have to integrate with Azure Cloud Services can be replaced with an extension method that hooks up to the Azure Configuration and environmental events for when the role is requesting to shut down, etc.A (hand-wavy) example of how this could work is by doing:
Provider implementation example
Notice the lack of the
Init
method (which in the current version receives the configuration property bag and a runtime to be able to resolve services (service location as opposed to dependency injection). The current approach also requires a parameterless constructor, whilst in the proposed version the provider is injected with its name, strongly typed options specific to that named provider, and any number of services coming from the container. Each provider is free to store theIOptions<TOptions>
and subscribe to changes if that is appropriate, otherwise they can just access and read the options at activation time, which will likely be how we do our first migration to the new system, since it's how it works today.Unit testing
Since we are getting rid of the serializable
ClusterConfiguration
, in order to set up the test cluster, developers will be encouraged to use a custom Startup type to configure it. TheTestCluster
infrastructure will support passing declarative settings to all silos in the cluster by leveraging the Configuration subsystem, so not all the configuration need to be statically determined by each silo individually.Legacy support of ClusterConfiguration
Even though the old
ClusterConfiguration
will no longer be used internally, we will initially provide a way to re-use this legacy class and map that declarative data to the new configuration system, since the new approach is more flexible. This is to ease the migration process for existing users. Nevertheless, we'll mark this approach as obsolete and eventually get rid of it. That means that users that do not care about the more flexible configuration could just do:There is no need to specify a Startup type when using this approach.
For custom extensions (such as custom Storage or Stream provider implementations), we'll defer the decision of whether will require changes to the implementation to align with the new approach, or whether we can provide a bridge so that the existing implementations are source code compatible and can be recompiled without any changes. Which option we choose will be based both on feedback from the community and the shape that this Builder approach takes.
Changes to stream providers dynamic reconfiguration
Short version: we'll still allow adding and removing stream providers dynamically at the silo-level, but not automatically propagate the changes to the entire cluster. Each silo should implement this at the application level (ie: by all polling an external store with this information and reacting similarly). Long version: This is a niche feature that we added not so long ago, but since there will be changes to it, I wanted to make them explicit. Currently there is a way to use the ManagementGrain to add new stream providers after the cluster started (or remove existing ones). This call receives an updated
ProviderCategoryConfiguration
(part of the legacy configuration oject) with the new stream providers included, and it serializes it and sends it to all the silos in the cluster. They would then deserialize it and add or remove the new/removed providers in the list (but wouldn't update if there's changes in the existing running providers). Since this is a strange feature that can have unforseen consequences if a particular silo misses an update (because it was restarted, etc) and requires app-level code to have configurations in sync, we are considering dropping the feature. We will still provide an API to add or remove stream providers at runtime, but this can be done at the silo level, not at the cluster level. That means that all silos in a cluster should be polling for a certain external config change that would trigger addition or removal of a stream providers if their domain requires so. This removes the constraint that the entire configuration for each stream provider implementation must be serializable, which we are moving away from with the builder approach.Working prototype
For the curious, I created a prototype that runs (although doesn't really hosts Orleans, just validates that configuration flows correctly to the right providers, and that the extension methods that take delegates are indeed achievable and not just hand-waving a design). The prototype references the Microsoft.AspNetCore.Hosting package to avoid copying a few of the hosting classes, but in the real implementation we intend not to do so to avoid confusion with namespaces and other non-Orleans related hosting abstractions, and just copy from their sources and adapt them. Ideally at some point we can converge if they provide a hosting abstraction not so tightly coupled to ASP.NET, as is being discussed here.
The prototype is in jdom/Orleans.Hosting.