asynkron / protoactor-dotnet

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
http://proto.actor
Apache License 2.0
1.71k stars 285 forks source link

Introduce infra and examples on sharding #1486

Open rogeralsing opened 2 years ago

rogeralsing commented 2 years ago

Akka Cluster Sharding is somewhat similar to virtual actors. instead of virtual actors, you have shards, which then, in turn, contain entities.

The cons of sharding are that it's not very fine-grained, you can get unbalanced workloads across nodes. The pros are that it is more lightweight on topology changes.

e.g. instead of moving millions of virtual actors around. you move far fewer shards. (in total there would be as many entities as virtual actors)

This would not load the PID Cache and identity lookup as much as virtual actors.

To build this, you would basically have a Shard virtual actor. that then routes messages to the correct child entity actor.

It could also take advantage of batching for message delivery. e.g. messages to entities in the same region could be grouped together and sent together, and potentially Acked back together. Where the shard region virtual actor could then use fire and forget messaging to the child entities.

image

rogeralsing commented 2 years ago

Like so

Skärmavbild 2022-02-26 kl  14 00 51
rogeralsing commented 2 years ago

I found this today: Akka can do Local Affinity processing of Kafka partitions: https://www.youtube.com/watch?v=Ad2DyOn4dlY

We should add info about this in the local affinity docs and describe the similarities/differences