search

dotnet / orleans

Cloud Native application framework for .NET
https://docs.microsoft.com/dotnet/orleans
MIT License
10.06k stars 2.03k forks source link

Atomic rollouts #8839

Closed verdie-g closed 7 months ago

verdie-g commented 8 months ago

In the paper Towards Modern Development of Cloud Applications by Google that gave birth to Service Weaver, it mentions

The closest solutions to our proposal are Orleans [74] and Akka [3]. These frameworks also use abstractions to decouple the application and runtime [...] None of these systems support atomic rollouts, which is a necessary component to fully address challenges C2-C5.

Orleans - Server configuration briefly mentions Green/Blue deployments

Within each cluster, all hosts must use the same ServiceId. Multiple clusters can share a ServiceId, however. This enables blue/green deployment scenarios where a new deployment (cluster) is started before another is shut down.

Does Orleans support atomic rollout through Green/Blue deployments?

ReubenBond commented 8 months ago

Hi @verdie-g, yes, Green/Blue deployments are the same thing as Service Weaver's atomic rollouts, as far as I can tell. If someone comes across this and believes that is not the case, I am interested to hear their perspective.

Orleans supports both Green/Blue and Rolling upgrades.

Green/Blue deployments were more common a decade or so ago, but there are various issues with that approach. For one, it does not dictate how external work is coordinated. For example, if you have a service which is not a stateless request/response RPC service, then there might need to be some coordination for the other work which it performs - such as processing messages from a message queue. Otherwise, you will find that you have multiple services performing the work, unaware of each other, which might produce observably incorrect behavior. A common place people would hit this in Orleans using Green/Blue deployments is with Reminders and with Streams. Rolling deployments solve that issue since you maintain a single logical cluster. The in-built persistence in Orleans at least prevents lost updates in case you have multiple activations of a grain (eg in different clusters), but it's not idea.

Another issue with Green/Blue is that you need to have twice the capacity (1x for the Blue cluster, 1x for the Green cluster) during the rollout and may need to pre-warm the new cluster whenever switching traffic over (it might be best to do this gradually & slowly).

Like so many things, there are trade-offs either way. Rolling deployments require that wire changes are backwards compatible and possibly forwards compatible. Orleans addresses these challenges:

I hope that helps. Apologies for the slow response.

verdie-g commented 8 months ago

Thanks for that detailed answer.

Are these green/blue drawbacks the reason why it is not more documented/pushed forward for Orleans?

allowing grains to migrate from one host to another when they encounter an incompatible message.

What is an incompatible message for a backward/forward compatible serialization? Can I find some documentation to learn more about that?

ReubenBond commented 8 months ago

Here is some more information: https://learn.microsoft.com/en-us/dotnet/orleans/grains/grain-versioning/grain-versioning