search

devsisters / shardcake

Sharding and location transparency for Scala
https://devsisters.github.io/shardcake/
Apache License 2.0
395 stars 30 forks source link

Singleton entities not being started after sharding.refreshAssignments #99

Closed finalchild closed 10 months ago

finalchild commented 10 months ago

Can this sequence of events cause a singleton entity not to spawn properly?

  1. The pod with shard 1 fails and is restarted with the same ip address.
  2. The pod calls sharding.refreshAssignments, which runs with .forkDaemon.
  3. Before the pod knows its original assignment, sharding.registerSingleton and sharding.register both runs. The pod thinks it has no assignment.
  4. sharding.refreshAssignments finishes
ghostdogpr commented 10 months ago

I think you are right, it is possible. It would only happen if the node dies instantly (not a clean stop) without calling unregister and it getting assignments is slower than going to the registerSingleton call.

I opened https://github.com/devsisters/shardcake/pull/100 to make sure we get the initial assignments before we register any singleton or entity.