State implementation details

[artemiusgreat]

Hi. Most of frameworks for distributed systems don't seem to have built-in in-memory storage. They're either stateless or provide some sort of a callback to persist state in the distributed storage, like Redis or Mongo.

Meanwhile, Orleans can act as a distributed cache right out of the box using in-memory storage. So, when I call GrainFactory.GetGrain() Orleans is supposed to create an instance of this grain on the REMOTE server in the cluster, not on the local client, and if this grain has some internal fields or properties, it will maintain the last value the was set.

Now, let's take this grain or this one for example.

var grain = GrainFactory.GetGrain<ICatalogTestGrain>(startingKey + i);

Eventually, this call lands here which seems to create LOCAL instance of the grain.

public GrainReference CreateReference(GrainId grainId)
{
  return (GrainReference) Activator.CreateInstance(
    type: _referenceType,
    bindingAttr: BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic,
    binder: null,
    args: new object[] { _shared, grainId.Key },
    culture: CultureInfo.InvariantCulture);
}

At the moment I can't figure this out. So far it seems that Orleans creates local instance of the grain and every time when I call grain's method, it's executed locally and then synchronized with the server side. Am I missing something?

Could somebody point me to the right source code or explain where exactly Orleans keeps in-memory state, a client or a server?

[nkosi23]

Most likely what is instantiated locally is not the Grain itself but a reference to the grain. This enables the Orleans runtime to intercept your method calls, serialize them and send them over to the Orleans server. Then these messages are queued server-side and executed serially by the grain in a first-in-first-processed basis.

The grain is activated server-side and the method is executed server-side. The method calls you make from the client are turned into messages sent to the server (the parameters are serialized etc...).

[artemiusgreat]

@nkosi23 Do you know where exactly in the code grain reference intercept local calls, and if the grain itself is created on the server side, how does it keep it's state between requests?

[nkosi23]

@artemiusgreat No unfortunately I do not know where exactly in the source code the interception is made, I have never felt a need to dig into this so far. In case this helps you find out: there is also some code generation going on. You can find more details on this page of the documentation.

As far as keeping state is concerned, the in-memory state of Grains is not stored for you automatically by the Orleans runtime, you need to be intentional. When a grain is activated the Orleans runtime will by default keep the grain in memory on the server until the grain is idle for 2 hours if I remember correctly (this delay is configurable). However this in-memory storage is not reliable as it will be lost once the grain is deactivated (it is useful for caching and to enable a wide range of other scenarios needing high-throughput).

If you need persistent/durable state, the easiest option is to use the built-in persistent state framework provided by Orleans, the documentation is here. However this convenience comes at the cost of flexibility, therefore depending on your needs you may instead want to manage persistence yourself by making database calls from within your grain methods. You can also combine both approaches.

[oising]

If you're using GrainFactory directly, you're already in a silo / orleans cluster. If you're actually remote, you'll be using Orleans.ClientBuilder to create an IClusterClient instance (which incidentally implements IGrainFactory to mirror the server-side interface.).

Some facts:

• Orleans uses a codegenerator that runs at build time to create proxy classes for all of your grains in the cluster.
• When you use IClusterClient, the grains are instantiated remotely and their state is remote, associated with whatever grain storage provider you have configured
• The local proxy and the server-side grain share your IGrain interface, but have different implementations
• Local grain calls are queued on the server to be executed against the remote grain and the return value (if any) is marshalled back to you over TCP

[artemiusgreat]

@nkosi23 @oising Thank you for answering and sorry for bothering, but I still would like to clarify how state maintenance works under the hood. I took this sample and modified it to have some simple internal IN-MEMORY state. No persistence. Just a class variable X in the grain that is somehow keeps its state in the memory.

Grain

using System.Threading.Tasks;
using Orleans;

namespace HelloWorld
{
  public interface IHelloGrain : IGrainWithStringKey
  {
    Task<int> SetA(int value);
    Task<int> GetA();
  }

  public class HelloGrain : Grain, IHelloGrain
  {
    private int x = 1;

    public Task<int> SetA(int input) => Task.FromResult(x = input);
    public Task<int> GetA() => Task.FromResult(x);
  }
}

Program

using System;
using HelloWorld;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Orleans;
using Orleans.Hosting; 

using var host = new HostBuilder()
  .UseOrleans(builder => builder.UseLocalhostClustering())
  .Build();

await host.StartAsync();

var grainFactory = host.Services.GetRequiredService<IGrainFactory>();
var friend = grainFactory.GetGrain<IHelloGrain>("friend");

Console.WriteLine("BEFORE: " + await friend.GetA());

var result = await friend.SetA(5);

Console.WriteLine("AFTER: " + await friend.GetA());
Console.ReadKey();

await host.StopAsync();

As you can see, the initial value of X is 1.

• Then, separate request SetA changes it to 5
• Then, separate request GetA requests the last value that was set, which is 5

So, there is no persistence, but the value 5 was maintained between 2 requests SetA and GetA. If there was no state maintenance, then request GetA would return the initial value, which is 1.

As far as you mentioned, everything should happen on the server only, so I'm wondering, how this variable kept its state of 5 between 2 requests? Does Orleans use anything like singletons and session management on the server side to create grain instance once and then use it in the following requests? I can't find the code that is actually creating the grain on the server and can't understand how the state was maintained.

[oising]

@artemiusgreat -- when we say "persistence", we're talking about hard-configured persistence, such as using Redis or Azure Storage as a backing storage provider. What you're doing is nothing more than setting a field on a grain which is currently loaded in memory. There is no reason for that grain to unload itself (or "deactivate" in Orleans parlance) because there is no memory pressure on the silo. If that grain were to unload implicitly or explicitly, or the silo to stop and restart, then it would lose the value on the field (just like a field on a class in a process/exe that is restarted.) Once a grain is instantiated ("activated") then it will remain in memory and any changes made to the fields on that class will remain until the grain either deactivates, or you shutdown the process.

I think you're attributing too much "magic" as to how Orleans works. Orleans persistence just means a way to serialize state to an external provider so the grain will remember your changes. Grain classes themselves are not implicit state - i.e. changing a field on the grain class is not considered "state" in the Orleans sense of the word. Grain state is an explicit thing, and usually involves creating a dedicated class to be serialized that holds fields that you want persisted between activation/deactivations of your grain instance. Hope this helps!

[nkosi23]

@artemiusgreat Also, if your question really is "where can I find the source code loading the grain instance and managing requests" keep in mind that what Orleans is doing is non-trivial. This is not as simple as storing a singleton in a static class. There is the notion of Grain Placement (Orleans dynamically places grains on the right server - for you say server but keep in mind that Orleans manages a cluster of silos), Request Scheduling, and a number other notions.

As far as I am concerned I am not a committer so I am only familiar with these notions from a user perspective, but not at the source code level. What I simply wanted to point here is that maybe you do not realize that what you are asking may actually be: how does the whole Orleans runtime works behind the hood. I mean, the behavior you are curious about is not just the result of some singleton storing some data somewhere, it is the result of various pieces of Orleans interacting together.

Maybe you are curious and want to understand the source code, in this case this absolutely doable, just keep in mind that this is non-trivial so first studying the documentation website thoroughly would be of the essence to allow you to find your way in the source code.

However, if all you want to know is: which assumptions and limitations do you need to keep in mind to use Orleans intelligently, the rough idea you need to keep in mind is that one the key value propositions of Orleans is that it allows people to develop cloud applications as if they developed stateful applications and their data was always kept in memory. If all you want to know is how can you use Orleans intelligently without making extremely wrong assumption, what @oising told you along with the documentation website should answer all the questions you are asking. As far as the source code is concerned, this is non trivial so you need to first understand Orleans concepts before you can find your way in the source code (and turn your questions into more specific questions).

[artemiusgreat]

@oising That's the thing. I never asked about persistence, only about objects in memory and their variables. I'm trying to understand how Orleans keep grain instances in memory on the server.

To rephrase my question, let's compare Orleans to Web API or MVC. A client connects to a server via HTTP. This is connectionless protocol, so the client sends HTTP request, server performs some action, closes HTTP connection, and shuts down. There are no objects left in memory when HTTP connection is closed, most of objects are transient.

[ApiController]
public class SomeController : ControllerBase
{
  int x = 1;
  [Route("set")] public int Set(int v) => x = v;
  [Route("get")] public int Get() => x;
}

I hit these URLs to check original value, set a new one, and check again if it's still in memory.

http://localhost:5000/get           // returns 1
http://localhost:5000/set?v=5    // returns 5
http://localhost:5000/get           // returns 1 because object are not retained in the memory

Orleans is also expected to make a remote call, but once some action is performed SetA, the grain keeps its state GetA and as far as I understand it keeps it on the server. So the question, how Orleans is different from the Web API and how it creates grain instances on the server so they don't get disposed after the previous request?

My understanding is that on the server side Orleans has some sort of an infinite loop or some cache, like session, that is always running. This cache is used to keep grain instances alive between requests, but I can't find the exact place in code that creates grain instance on the server.

[artemiusgreat]

@nkosi23 Exactly, I need to know how it works under the hood. It's not a question about how to use it, it's a question of what's happening at the point when specific Silo server in the cluster was chosen to host the grain instance. How does server creates this instance so it stays in memory for specific period of time?

[oising]

@artemiusgreat -- back in the days before DI and the associated explicit lifecycles (transient, scoped, singleton) , you'd have gotten a different result for your tests for persisting a field between web requests - ASP.net would share instances of your page classes between requests and this could cause issues if you weren't aware of it. But since MVC controllers are transient these days, you get a new instance for every http request, but their injected dependencies could be scoped differently.

Grains also have an associated lifecycle which is talked about briefly here: https://dotnet.github.io/orleans/docs/grains/grain_lifecycle.html but frankly, if you want to understand exactly how it's all done from a technical perspective, then get digging into the source. But don't confuse yourself by comparing grains to controllers -- grains have a strong identity:

and controllers do not.

Read more here: https://dotnet.github.io/orleans/docs/index.html

[artemiusgreat]

Oisin's answer reminded me that routes and controllers is just one of the tool built into Kestrel server. Creating my own middleware I can easily separate scoped instances created per request and singletons. https://docs.microsoft.com/en-us/aspnet/core/fundamentals/middleware/

Example

public class SomeClass
{
  public int State { get; set; } = 0;
}

public class Startup
{
    public void Configure(IApplicationBuilder app)
    {
        var singletonInstance = new SomeClass();  // initialized only once

        app.Run(async context =>
        {
            var scopeInstance = new SomeClass();  // initialized on every request

            Console.WriteLine("Scope : " + scopeInstance.State);  // always prints 0 per request
            Console.WriteLine("Singleton : " + singletonInstance.State);  // shows 0, then changes to 5 permanently

            scopeInstance.State = 5;
            singletonInstance.State = 5;

            // I can also use session to change singleton based on user and make singleton act like a scope

            if (context.Session.Id == "SomeUser") 
            {
                singletonInstance.State = new Random().Next();
            }

            await context.Response.WriteAsync("Demo");
        });
    }
}