[Proposal] Provide ASP.NET service providers written in Orleans

[bradygaster]

[This is a work-in-progress and will evolve a bit]

The ASP.NET developer faces a myriad of challenges the moment their web apps are scaled horizontally. The addition of horizontally-scaled compute layers - hardware or PaaS-based layers - results in additional considerations. By distributing one's application across multiple layers, performance is increased, but so is complexity.

• Session state becomes harder to manage because one never knows which server they'll land on.
• Distributed caching providers.
• Distributed data protection issues (this is a stretch but, maybe Orleans could be appropriate here).

At the moment, we're investigating the idea of shipping a few default implementations of interfaces like IDistributedCache (doc) or to help with distributed session providers and/or Blazor session state management. Another common scenario is a custom SignalR backplane in which Azure SignalR Service isn't appropriate or an on-premise solution is needed.

We'd like your input

We're confident there are other scenarios in which ASP.NET developers face issues the moment they enter into a distributed landscape or a cloud-native deployment scenario in which the compute layer could be transient, like a K8S pod. If you've experienced challenges once you scale your ASP.NET app out with these or other cases, we'd be interested in your thoughts, so leave a note down below.

Our goal is to ship, with the .NET 8.0 release, a set of providers for building distributed ASP.NET apps running on distributed compute layers built atop Orleans. So help us figure out which of these scenarios we should tackle in our first wave of including Orleans as a key ingredient to the ASP.NET stack.

[ElanHasson]

That is a really awesome idea! In addition to the mentioned providers, it would be awesome to have providers for:

• Identity provider (UserStore/RoleStore and friends) (Was not fun doing implementing them a few years back)
• File Providers (would be awesome for CMS / Cache Use cases)
• DataProtection provider (let me take load of my DB)
• FeatureManagement provider

[rjygraham]

Settings provider/coordinator (depending on how settings are configured, its possible settings get out of sync across nodes for brief periods of time). This is correlated to @ElanHasson mention of FeatureManagement provider which I'm assuming is something akin to feature flags.

[bradygaster]

@ElanHasson appreciate your list! Data Protection is one I'd included above as well [currently experiencing some pain on this one personally]. I'd be interested in maybe how Orleans could solve some of these problems, conceptually.

[ghost]

We're moving this issue to the 4.0-Planning milestone for future evaluation / consideration. Because it's not immediately obvious that this is a bug in our framework, we would like to keep this around to collect more feedback, which can later help us determine the impact of it. We will re-evaluate this issue, during our next planning meeting(s). If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.

[bradygaster]

Changed title from "default" to "provide" as we wouldn't change any defaults, just provide Orleans-backed options. Defaults is a bit more aggro than I'd hoped to propose.

[halter73]

Here's a more comprehensive list of ideas. Many were already mentioned above:

• Distributed "soft" state (IDistributedCache)
• Session state (ISessionStore)
  • Given that the DistributedSessionStore is already based on IDistributedCache, is it better to just to implement that?
• Signalr "backplane" (HubLifetimeManager)
• Distributed output caching (IOutputCacheStore)
  • Similar to IDistributedCache with tag-based eviction and segmented buffers for output caching
  • https://github.com/dotnet/aspnetcore/issues/41955 @sebastienros
• Something for distributed rate limiting ( ???)
  • Currently planning on implementing a RateLimiter or PartitionedRateLimiter<string> directly using StackExchange.Redis, but it might be better to use Orleans.
  • Or possibly to build a new and improved version of IDistributedCache (possibly a genericized and renamed IOutputCacheStore?) that is then implemented using Orleans. Maybe even an improved ISessionStore could use it.
  • https://github.com/dotnet/aspnetcore/issues/41861

For the following I wonder if it's worth it if extra configuration is needed to make the state truly persistent instead of "soft" state. Would many developers really want "soft" implementations of these abstractions? Or is it just adding an extra step to truely persisting keys/files/identites?

• Data protection providers (IDataProtectionProvider)
  • If extra configuration is needed to make the state truly persistent instead of "soft" state, is this worth it?
• File providers (IFileProvider)
  • I have the same question about whether developers would want a "soft" IFileProvider or if there should be a different abstraction.
• Identity providers (RoleStoreBase, UserStoreBase)

[ElanHasson]

@bradygaster I can see Orleans being used for keys that get very hot in terms of access (a cache)

As an aside, it would be cool to have Data protection for grain state by wrapping existing storage providers :)

[bradygaster]

cc @glennc as we were talking about this earlier

[bradygaster]

As an aside, it would be cool to have Data protection for grain state by wrapping existing storage providers :)

I don't hate that idea at all. cc @blowdart to tell me not to think about it.

[rwkarg]

Would like to see: Rate Limiting (currently using an Orleans/TokenBucket based solution) YARP Health Checks (https://github.com/microsoft/reverse-proxy/issues/1723#issuecomment-1128239568)

Periodic Tasks - Things that need to happen on some regular period (every 4 hr.) or a CRON schedule (19:00 UTC on Fridays) but shouldn't have concurrent processing. Orleans bring availability by utilizing Reminders for durable triggering as well as ensuring one active grain at a time, and provides efficiency by being able to get deactivated between triggers.

[bradygaster]

love those ideas.

[ElanHasson]

@rwkarg, check out https://github.com/web-scheduler/ for the Cron Scheduler (I run this project).

For distributed rate limiting, @ReubenBond made something for Orleans 4 + .NET 7: https://github.com/ReubenBond/DistributedRateLimiting.Orleans

[seniorquico]

Just passed by this thread, and I'll attach my 👍. I've been part of a few teams that have built large-scale applications with ASP.NET Core & Orleans, and Orleans has been great scaling out some ASP.NET Core capabilities.

I find the caching suggestions interesting. We shifted to an Orleans-first development model, and we haven't used IDistributedCache since. Our cluster/grains already act as a strongly-typed, distributed, read-through cache with sliding TTLs (based on the grain activation timeout). Further, we've been able to exploit stateless workers & [Immutable] models in co-hosted ASP.NET Core + silo deployments (in a heterogenous cluster) to transparently make specific data local. I previously saw Polly's support for read-through response caching based on IDistributedCache. I hadn't seen mention of the new IOutputCacheStore, though. I guess we just haven't hit the need to reach this level of optimization? Even still, I can't imagine an Orleans-powered implementation of these interfaces would be too difficult to provide for plug-and-play operations.

Here are some things we already found success with an Orleans-powered solution:

• Feature Management: We previously used a custom flag/dynamic flag provider backed by Orleans grains and Azure Table Storage.
• Identity: We currently use an IUserStore/IUserRoleStore backed by Orleans grains and Azure Table Storage (of note, we didn't implement the IQueryableUserStore/IQueryableRoleStore capability). In addition to powering auth workflows, we also leverage the common abstractions in Microsoft.Extensions.Identity.Core in other "microservices" running in our cluster.
• Options: We currently use a custom IOptionsMonitor to provide change notifications through Orleans streams (or maybe it was an IChangeToken, I don't recall exactly).
• SignalR: We previously used a custom backplane powered by Orleans grains and streams.
  • There's already a project for this: https://github.com/OrleansContrib/SignalR.Orleans

And not really an ASP.NET Core thing... but a fun idea our team had kicked around a while back. We use HttpClientFactory with Polly extensively. There are some interesting failure scenarios that we thought would benefit from a distributed circuit-breaker. What better to power it than Orleans? 😁 Our discussion was inspired by this project: https://github.com/Polly-Contrib/Polly.Contrib.AzureFunctions.CircuitBreaker

[ElanHasson]

I love the distributed circuit breaker concept!