dotnet / orleans

Cloud Native application framework for .NET
https://docs.microsoft.com/dotnet/orleans
MIT License
10.08k stars 2.03k forks source link

[Epic]: Support Multi-clustering for Orleans #7485

Open rafikiassumani-msft opened 2 years ago

rafikiassumani-msft commented 2 years ago

We might need to add documentation on why this might not be potentially needed for certain use cases.

ReubenBond commented 2 years ago

We discussed this with an internal partner team today and have some ideas for how to implement this well. I'm documenting some of the takeaways of the discussion here for our own good.

I suggest we call this feature "metaclusters" because it involves a cluster of clusters (a cluster is a set of collaborating processes, this is a set of sets of processes).

We want to support communication between multiple Orleans clusters, typically geographically separated.

This will involve some changes to Orleans' core and we have a few guiding principles:

Our current thinking in terms of design and work:

Metacluster Membership

Routing/Networking

Placement:

Non-goals:

cc @JohnMorman @juyuz

JesOb commented 2 years ago

Does that mean orleans will able to run with ASP .Net Core in Google App Engine Flexible?

Magazin80 commented 1 year ago

Is multi-clustering supported in the latest release v3.X? Documentation says that multi-clustering was removed in v2. Does it mean that we should not plan for it for now? Thanks!

ElanHasson commented 1 year ago

@Magazin80, no mult-clustering in v3.x and as of yet v7.0 (formerly 4.0) . I'll defer to @ReubenBond on the timeline.

jan-johansson-mr commented 1 year ago

Hi @ReubenBond and @ElanHasson,

The support for metaclusters seems good to me, that is, in my mind, a client can use a cluster meta ID that points to a collection of clusters, where the destination Silo may be in any of the "physical" clusters. But I am reacting to a wording in the proposal, and the wording is pay as you go. Does that means that this metaclusters feature, if done, only is applicable to Azure?

So far, I've been able to use Orleans on-prem and with azure solution, but is the direction of some of the features of Orleans to be locked-in to Azure in the future? This worries me. Another such feature is the transactional feature, where the transactional provider given out-of-the-box in Orleans only works on Azure, there is no provider given, out-of-the-box, that works on ADO. Of course, one can write such a provider, and I did. However, this metaclusters feature, if tailored only for Azure, is something completely different, if this is the intended direction.

Kindly

0x53A commented 1 year ago

[...] the wording is pay as you go. Does that means that this metaclusters feature, if done, only is applicable to Azure?

I assume the term was meant on a technical level, not a billing level. Typically, pay as you go means that if you don't use the feature, you don't "pay" for it in regards to performance, complexity, etc.


From my side, I'd be really interested in this for connecting Cloud (typically, but not necessarily, Azure) and On-Premise.

Currently we use a WCF Server in the cloud, a WCF Client on-premise, and then run duplex communication through a persistent Websocket using the net.http Binding and Client Callbacks.

We would be interested in modernizing this solution. The complexity comes from the fact that the On-Premise client / cluster / whatever you want to call it, is NOT directly internet addressable. Instead it will initiate the connection itself, and then everything needs to be routed through this one persistent connection. If the connection breaks, only the On-Premise part can re-establish the connection.

ReubenBond commented 1 year ago

[...] the wording is pay as you go. Does that means that this metaclusters feature, if done, only is applicable to Azure?

I assume the term was meant on a technical level, not a billing level. Typically, pay as you go means that if you don't use the feature, you don't "pay" for it in regards to performance, complexity, etc.

Yes, precisely.

The complexity comes from the fact that the On-Premise client / cluster / whatever you want to call it, is NOT directly internet addressable.

In your case, would a solution based upon a VPN be sufficient?

0x53A commented 1 year ago

In your case, would a solution based upon a VPN be sufficient?

No, we have multiple customers running our software in their on-premise, with the cloud portion hosted by us. We don't really want to setup hundreds of VPNs. That's why using http/websockets is so valuable (compared to raw TCP), it's easy to allow-list on the customers firewall and easy to route through nginx/app gateway.

ReubenBond commented 1 year ago

No, we have multiple customers running our software in their on-premise, with the cloud portion hosted by us.

That is a non-goal of Orleans: it is not designed to allow untrusted third parties to connect directly to an Orleans cluster. For that, you should use a gateway, eg based on HTTP APIs, SignalR, gRPC, etc.

KSemenenko commented 1 year ago

My case - there is a cluster in the US and a cluster in the EU, in each cluster WebAPI client and Silo. Data is not distributed across regions. What to do if the user from the US will connected to the EU cluster? then i would like to be able to webapi from EU make connecttion with US Silo and servce client

ReubenBond commented 1 year ago

@KSemenenko, is your case addressed by the Orleans.MultiClient repo? Admittedly, that needs to be updated to support Orleans 7

StephenStrickland commented 9 months ago

Any chance this is on the radar for this year?