How can grains be loaded as plugins in Orleans 7?

[Tim-Utelogy]

I'm an Orleans newb and I'm trying to get grains to load as plugins. I suspect this was easy with 3.0 via ConfigureApplicationParts because you could explicitly tell it where to load an assembly from. However, the version migration documentation states:

Remove calls to ConfigureApplicationParts. Application Parts has been removed. The C# Source Generator for Orleans is added to all packages (including the client and server) and will generate the equivalent of Application Parts automatically.

This works fine when exe project has a direct reference to a grain containing project, but for plugins, the exe project should have no reference to anything other than the required orleans server package(s). This would seem to imply that I need to use another library to find/load the plugin grain assemblies before calling UseOrleans but when I do that I get an exception that it can't find the assembly that I already loaded. How can I go about loading grains as plugins?

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

using Orleans.Configuration;

using Weikio.PluginFramework.Catalogs;

int instanceId = 0; //make this a command line arg, IConfiguration or both?

var path = @"C:\Users\lanat\source\experiments\Orleans\Grains1\bin\Debug\net7.0";
var pluginLoadContextOptions =
    new Weikio.PluginFramework.Context.PluginLoadContextOptions
    {
        AdditionalRuntimePaths = new List<string> { path },
        UseHostApplicationAssemblies = Weikio.PluginFramework.Context.UseHostApplicationAssembliesEnum.Always,
    };
var folderOptions =
    new FolderPluginCatalogOptions()
    {
        IncludeSubfolders = true,
        SearchPatterns = new List<string> { "*grain*.dll" },
        PluginLoadContextOptions = pluginLoadContextOptions,
    };

var folderCatalog = new FolderPluginCatalog(path, type => type.Implements<IGrainBase>(), folderOptions);
await folderCatalog.Initialize(); //load the assemblies

await Host.CreateDefaultBuilder(args)
    .ConfigureAppConfiguration(configHost => configHost.AddUserSecrets<Program>())
    .UseOrleans((hostContext, siloBuilder) =>
        siloBuilder
            //Example of how to use localhost clustering. Its important to note that you need to specify the master node endpoint if this is not the master node
            //https://github.com/dotnet/samples/blob/80f6cdfa676558ef6038f92633a67944b6427d76/orleans/TicTacToe/Program.cs#L10
            .UseLocalhostClustering(
                siloPort: EndpointOptions.DEFAULT_SILO_PORT + instanceId,
                gatewayPort: EndpointOptions.DEFAULT_GATEWAY_PORT + instanceId/*, serviceId: "ClusterNode1", clusterId: "dev",*/
                /*primarySiloEndpoint : new IPEndPoint(IPAddress.Loopback, EndpointOptions.DEFAULT_SILO_PORT)*/)
            .ConfigureLogging(logging => logging.ClearProviders().AddConsole()))

    .UseWindowsService()
    .Build()
    .RunAsync();

at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, RuntimeAssembly requestingAssembly, Boolean throwOnFileNotFound) at System.Reflection.Assembly.Load(AssemblyName assemblyRef) at Orleans.Serialization.ReferencedAssemblyHelper.gExpandAssembly|5_3(HashSet1 assemblies, Assembly assembly) at Orleans.Serialization.ReferencedAssemblyHelper.<GetApplicationPartAssemblies>g__ExpandApplicationParts|5_1(IEnumerable1 assemblies) at Orleans.Serialization.ReferencedAssemblyHelper.GetApplicationPartAssemblies(Assembly assembly) at Orleans.Serialization.ReferencedAssemblyHelper.AddAssembly(HashSet1 parts, Assembly assembly) at Orleans.Serialization.ReferencedAssemblyHelper.GetRelevantAssemblies(IServiceCollection services) at Orleans.Serialization.ServiceCollectionExtensions.AddSerializer(IServiceCollection services, Action1 configure) at Orleans.Hosting.DefaultSiloServices.AddDefaultServices(IServiceCollection services) at Orleans.Hosting.SiloBuilder..ctor(IServiceCollection services) at Microsoft.Extensions.Hosting.GenericHostExtensions.AddOrleans(IServiceCollection services) at Microsoft.Extensions.Hosting.GenericHostExtensions.<>cDisplayClass1_0.b0(HostBuilderContext context, IServiceCollection services) at Microsoft.Extensions.Hosting.HostBuilder.InitializeServiceProvider() at Microsoft.Extensions.Hosting.HostBuilder.Build() at Program.<

$>d0.MoveNext() in C:\Users\lanat\source\experiments\Orleans\ClusterNode1\Program.cs:line 31

[mitchdenny]

With the source generator all the assemblies are going to have to be reachable at compile time.

[Tim-Utelogy]

For clarification, do you specifically mean that the grain containing assemblies have to be reachable from the executable project when it is compiled? So there is currently no support for plugins and none planned? That seems like big impediment to heterogenous clustering where you don't want to bring all your grain types with you.

[mitchdenny]

I'll tag @ReubenBond for once he is back to answer definitively on that question.

I would say that plugin systems like that on the server side are a bit less common these days given the way apps are deployed in immutable ways.

[SebastianStehle]

I would not expose grains as plugins. I would make a grain plugin. The reason is that you want to have a loose coupling between application and plugins. e.g. if you want to replace Orleans or just upgrade it, you don't want to update all your plugins.

[Tim-Utelogy]

@SebastianStehle, I'm not sure I follow. Are you suggesting the silo and grains all be done as a monolith where the grains take dependencies on injected plugin classes that may or may not be loaded? If you have a simple code example that might help my understanding.

My overarching goal is for the same executable to load different sets of grains, that follow certain conventions, based on configuration. Different running instances of that executable could then run as a heterogenous cluster with the exe having a different support lifecycle than its plugins.

[SebastianStehle]

If you want to use a plugin you have to expose an interface. So instead of exposing the IPluginGrain interface I would expose IPlugin interface and keep that dependency free.

Lets say you have the following plugin:

interface ICalcPlugin
{
  int Calculate(int a, int b);
}

Then I would also define a grain for that:

interface ICalcPluginGrain
{ 
   Task<int> Calculate(int a, int b);
}

Then you need a plugin registry, where each plugin also gets an idea, and then you could implement the grain like this:

// PSEUDO CODE
class CalcPluginGrain : Grain, ICalcPluginGrain
{
   private readonly IPlugin plugin;

   public PluginGrain(IActivationContext context, IPluginManager pluginManager)
   { 
      plugin = pluginManager.GetPlugin<IPlugin>(context.Identity)
   }

   Task<int> Calculate(int a, int b) {
       return Task.FromResult(plugin.Calculate(a, b));
   }
}

Usually you also have some kind of ICalcManager:

interface ICalcManager
{ 
   Task<int> Calculate(int a, int b);
}

that is then implemented like this:

class CalcManager : ICalcManager
{
      Task<int> Calculate(int a, int b)
      {
           int result = 0;
           foreach (var id in pluginManager.GetIds<ICalcPlugin>())
           {
              var grain = grainFactory.GetGrain(id);
              result += await grain.Calculate(a, b);
           }
           return result;
      }
}

with this approach Orleans is an implementation detail.

[ReubenBond]

Adding to what others have said, you can achieve this in Orleans 7.x just as you could in 3.x. It's done a little differently: you would add the relevant assemblies explicitly during startup, like this:

services.AddSerializer(options => options.AddAssembly(asm));

I'm going to get into the weeds now, so disregard the rest of this if you're not interested in going there.

It looks odd to do it that way (why is it in an AddSerializer call?), so I will explain. It's not the intention that you should need to do this. Eg, see Sebastian's comments above, and the quoted message in the original post & what Mitch said about the expectation that everything is available at build time - Application Parts were a huge usability issue for developers. In addition, support for dynamically loading assemblies in .NET Core and later is deemphasized compared to .NET Framework, even if .NET Core introduces AssemblyLoadContext.

The reason that this lives in the AddSerializer call is that the source generator in Orleans is not Orleans-specific, it's Orleans.Serialization-specific, and Orleans.Serialization is not tied to the core of Orleans. The serialization library includes the core RPC bits, and the Orleans libraries use configuration to instruct it to generate proxies & invokers for grains using attributes on the relevant types:

[GenerateMethodSerializers(typeof(GrainReference))]
public interface IAddressable { } // IGrain inherits from IAddressable, so this is what causes Orleans to generate code for your grain interfaces

/// <summary>
/// This is the base class for all grain references.
/// </summary>
[Alias("GrainRef")]
[DefaultInvokableBaseType(typeof(ValueTask<>), typeof(Request<>))]
[DefaultInvokableBaseType(typeof(ValueTask), typeof(Request))]
[DefaultInvokableBaseType(typeof(Task<>), typeof(TaskRequest<>))]
[DefaultInvokableBaseType(typeof(Task), typeof(TaskRequest))]
[DefaultInvokableBaseType(typeof(void), typeof(VoidRequest))]
[DefaultInvokableBaseType(typeof(IAsyncEnumerable<>), typeof(AsyncEnumerableRequest<>))]
public class GrainReference : IAddressable, IEquatable<GrainReference>, ISpanFormattable
{

The extensibility goes further than that, eg, the transactions library customizes the invokable base type for transactional methods, but I will stop here for now.

[andreabalducci]

We have a plugins folder and a custom AssemblyLoader. Some plugins are compiled on project build, others are compiled at runtime (Roslyn) to support c# scripting scenario (we unload and reload newly compiled assembly on the fly).

With on-the-fly compiled assembly all worked fine. The process crashes with statically compiled plugins, same load error of this issue.

We spent almost an hour to figure out how Orleans 8 load application parts dynamically and fix the issue, reporting here hoping will help someone else.

Issue is in ReferencedAssemblyHelper.GetApplicationPartAssemblies

            return ExpandApplicationParts(
                new[] { assembly }.Concat(assembly.GetCustomAttributes<ApplicationPartAttribute>()
                    .Select(name => Assembly.Load(new AssemblyName(name.AssemblyName)))));

Assembly.Load fails.

Plugins are Orleans agnostic, so there's no need to load as application parts. Orleans has checks in place, every assembly is checked for ApplicationPartAttribute, if missing the assembly is ignored. We checked with DotPeek our assemblies and found ApplicationPart in the manifest, due to a depedendency with a "shared services assembly" carrying an unneeded Microsoft.Orleans.Sdk reference.

Fixed removing the unneeded Microsoft.Orleans.Sdk reference from our services assembly.

[SunSi12138]

I would like to suggest adding a functionality similar to the ApplicationPart in MVC to Orleans. Although there was something similar in the old Orleans, the functionality was different.

Imagine having many different types of loads and potentially specific locations (instead of having all Grains in one program). Currently, for different load types, we only reference the required Grain libraries and then package them into specific artifacts. However, if we could add them dynamically, we would only need to package one application base and then distribute Grain loads dynamically through the management interface. This approach would be extremely useful in clusters or microservices.

For example, in a large-scale e-commerce system, different Grain libraries could handle various tasks such as order processing, inventory management, and customer service. With the ability to add these Grain libraries dynamically, the system could adapt more flexibly to changes in load and business requirements.

Another example could be in a financial application where different Grain libraries are needed for risk assessment, transaction processing, and fraud detection. Dynamic addition would enable the application to update and expand its functionality without the need for a complete redeployment.