As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
As we discovered during the course of our pains-taking bug investigation, these were, in fact, the same issue:
The ClusterSingletonManager is supposed to always belong on the oldest node of a given role type, but an original design error from the time Akka.Cluster.Tools was first introduced to Akka.NET meant that nodes were always sorted in descending order of UpNumber. This is backwards: nodes should always be sorted in ascending order of UpNumber - this means that the oldest possible node is always at the front of the "who is oldest?" list held by the ClusterSingletonManager. This explains why the singleton could appear to move early during deployments and restarts.
The ClusterSingletonManager was suspectible to a race condition where if nodes were shutdown and restarted with the same address in under 20 seconds, the default "down removal margin" used by the ClusterSingletonManager to tolerate dirty exits, it would be possible after multiple successive, fast, restarts for multiple instances of the singleton to be alive at the same time (for a short period.)
Both of these varieties of problem, duplicate singletons, is what lead to duplicate shards.
Akka.Discovery and ClusterClient Discovery Support
In Akka.NET v1.5.27 we've added support for using Akka.Cluster.Tools.ClusterClient alongside with Akka.Discovery plugins to automatically discover the initial contacts you need for ClusterClientReceptionist instances in your environment.
Aaronontheweb
commented
1 month ago
Bumps
AkkaVersionfrom 1.5.22 to 1.5.27.1. UpdatesAkka.Persistence.Query.Sqlfrom 1.5.22 to 1.5.27.1Release notes
Sourced from Akka.Persistence.Query.Sql's releases.
... (truncated)
Changelog
Sourced from Akka.Persistence.Query.Sql's changelog.
... (truncated)
Commits
55827b7Update RELEASE_NOTES.md for 1.5.27.1 (#7306)3e2d49cFix remoting logging DefaultAddress race condition (#7305)3eacb6cAdded v1.5.28 placeholderb34d645V1.5.27 release notes (#7303)6a81fd3Akka.Cluster.Tools: deprecate ClustersSingletonManagerSettings.ConsiderAppVer...7180810Akka.Cluster.Tools: fix mutability and oldest state bugs with `ClusterSinglet...6b6afe4Add AlsoTo downstream failure propagation support (#7301)7811504Fix ActorMaterializerImpl null LogSource (#7300)f2e81c5Akka.Cluster.Tools.Singleton / Akka.Cluster.Sharding: fix duplicate shards ca...f4501e8migratedClusterSingletonManagertoswitchstatements (#7296)Updates
Akka.Persistence.Sql.Commonfrom 1.5.22 to 1.5.27.1Release notes
Sourced from Akka.Persistence.Sql.Common's releases.
... (truncated)
Changelog
Sourced from Akka.Persistence.Sql.Common's changelog.
... (truncated)
Commits
55827b7Update RELEASE_NOTES.md for 1.5.27.1 (#7306)3e2d49cFix remoting logging DefaultAddress race condition (#7305)3eacb6cAdded v1.5.28 placeholderb34d645V1.5.27 release notes (#7303)6a81fd3Akka.Cluster.Tools: deprecate ClustersSingletonManagerSettings.ConsiderAppVer...7180810Akka.Cluster.Tools: fix mutability and oldest state bugs with `ClusterSinglet...6b6afe4Add AlsoTo downstream failure propagation support (#7301)7811504Fix ActorMaterializerImpl null LogSource (#7300)f2e81c5Akka.Cluster.Tools.Singleton / Akka.Cluster.Sharding: fix duplicate shards ca...f4501e8migratedClusterSingletonManagertoswitchstatements (#7296)Updates
Akka.Persistence.Sql.TestKitfrom 1.5.22 to 1.5.27.1Release notes
Sourced from Akka.Persistence.Sql.TestKit's releases.
... (truncated)
Changelog
Sourced from Akka.Persistence.Sql.TestKit's changelog.
... (truncated)
Commits
55827b7Update RELEASE_NOTES.md for 1.5.27.1 (#7306)3e2d49cFix remoting logging DefaultAddress race condition (#7305)3eacb6cAdded v1.5.28 placeholderb34d645V1.5.27 release notes (#7303)6a81fd3Akka.Cluster.Tools: deprecate ClustersSingletonManagerSettings.ConsiderAppVer...7180810Akka.Cluster.Tools: fix mutability and oldest state bugs with `ClusterSinglet...6b6afe4Add AlsoTo downstream failure propagation support (#7301)7811504Fix ActorMaterializerImpl null LogSource (#7300)f2e81c5Akka.Cluster.Tools.Singleton / Akka.Cluster.Sharding: fix duplicate shards ca...f4501e8migratedClusterSingletonManagertoswitchstatements (#7296)Updates
Akka.Persistence.TCKfrom 1.5.22 to 1.5.27.1Release notes
Sourced from Akka.Persistence.TCK's releases.
... (truncated)
Changelog
Sourced from Akka.Persistence.TCK's changelog.
... (truncated)
Commits
55827b7Update RELEASE_NOTES.md for 1.5.27.1 (#7306)3e2d49cFix remoting logging DefaultAddress race condition (#7305)3eacb6cAdded v1.5.28 placeholderb34d645V1.5.27 release notes (#7303)6a81fd3Akka.Cluster.Tools: deprecate ClustersSingletonManagerSettings.ConsiderAppVer...7180810Akka.Cluster.Tools: fix mutability and oldest state bugs with `ClusterSinglet...6b6afe4Add AlsoTo downstream failure propagation support (#7301)7811504Fix ActorMaterializerImpl null LogSource (#7300)f2e81c5Akka.Cluster.Tools.Singleton / Akka.Cluster.Sharding: fix duplicate shards ca...f4501e8migratedClusterSingletonManagertoswitchstatements (#7296)Updates
Akka.TestKit.Xunit2from 1.5.22 to 1.5.27.1Release notes
Sourced from Akka.TestKit.Xunit2's releases.
... (truncated)
Changelog
Sourced from Akka.TestKit.Xunit2's changelog.