Open Aaronontheweb opened 2 years ago
HOCON configuration for this reproduction:
akka{
cluster.sharding{
# use separate journal implementations for Akka.Cluster.Sharding
journal-plugin-id = akka.persistence.journal.sharding
snapshot-plugin-id = akka.persistence.snapshot-store.sharding
state-store-mode = persistence
remember-entities-store = eventsourced
remember-entities = on
passivate-idle-entity-after = off
fail-on-invalid-entity-state-transition = on
}
persistence{
journal{
sharding{
# qualified type name of the SQL Server persistence journal actor
class = "Akka.Persistence.SqlServer.Journal.SqlServerJournal, Akka.Persistence.SqlServer"
# connection string used for database access
connection-string = ""
# SQL server table corresponding with persistent journal
table-name = ShardingEventJournal
metadata-table-name = ShardingMetadata
# should corresponding journal table be initialized automatically
auto-initialize = on
event-adapters {
tagger = "SqlSharding.Shared.MessageTagger, SqlSharding.Shared"
coordinator-migration = "Akka.Cluster.Sharding.OldCoordinatorStateMigrationEventAdapter, Akka.Cluster.Sharding"
}
event-adapter-bindings = {
"System.Object" = tagger
"Akka.Cluster.Sharding.ShardCoordinator+IDomainEvent, Akka.Cluster.Sharding" = coordinator-migration
}
}
}
snapshot-store{
sharding{
# qualified type name of the SQL Server persistence journal actor
class = "Akka.Persistence.SqlServer.Snapshot.SqlServerSnapshotStore, Akka.Persistence.SqlServer"
# connection string used for database access
connection-string = ""
# SQL server table corresponding with persistent journal
table-name = ShardingSnapshotStore
# should corresponding journal table be initialized automatically
auto-initialize = on
}
}
}
}
This is supposed to be stage 1 of the Akka.NET v1.5 migration.
In case it matters: both #6132 and this issue were reproduced using Akka.Persistence.SqlServer.
Changing the sharding configuration to read:
akka.cluster.sharding{
state-store-mode = ddata
remember-entities-store = eventsourced
}
Worked just fine and successfully instantiated all remembered entities and shards, but it still produced #6132
Can verify that the entities were all started via pbm
:
λ pbm actor hierarchy -s akka://SqlSharding/ -d 6
/
//system
//system/akka.persistence.journal.sharding
//system/akka.persistence.journal.sql-server
//system/akka.persistence.snapshot-store.sharding
//system/akka.persistence.snapshot-store.sql-server
//system/cluster
//system/cluster/core
//system/cluster/core/daemon
//system/cluster/core/daemon/heartbeatSender
//system/cluster/core/publisher
//system/cluster/heartbeatReceiver
//system/clusterEventBusListener
//system/deadLetterListener
//system/endpointManager
//system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FSqlSharding%40localhost%3A7918-1
//system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FSqlSharding%40localhost%3A7918-1/endpointWriter
//system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FSqlSharding%40localhost%3A7918-1/endpointWriter/endpointReader-akka.tcp%3A%2F%2FSqlSharding%40localhost%3A7918-1
//system/EventStreamUnsubscriber-1
//system/IO-TCP
//system/IO-TCP/$a
//system/IO-TCP/$a/$a
//system/log1-DefaultLogger
//system/recoveryPermitter
//system/remote-deployment-watcher
//system/remote-watcher
//system/remoting-terminator
//system/sharding
//system/sharding/hostReplicator
//system/sharding/hostReplicator/durableStore
//system/sharding/products
//system/sharding/products/0
//system/sharding/products/0/791f0f68-5f57-4be4-9e08-24e66fe89e4e
//system/sharding/products/0/RememberEntitiesStore
//system/sharding/products/10
//system/sharding/products/10/7a4a4fb4-85ce-41a1-a89d-015c41bd2e0f
//system/sharding/products/10/9e33a032-ce18-4922-85f7-6abfa0d23d6a
//system/sharding/products/10/RememberEntitiesStore
//system/sharding/products/11
//system/sharding/products/11/bfbdd6aa-7942-488f-8d74-cbf43b99d55f
//system/sharding/products/11/RememberEntitiesStore
//system/sharding/products/12
//system/sharding/products/12/2b16d499-f41c-4196-a7a3-78a148153158
//system/sharding/products/12/RememberEntitiesStore
//system/sharding/products/14
//system/sharding/products/14/402d3db2-2145-4e4a-b094-f1bb06c0d984
//system/sharding/products/14/7226945c-61a8-47e7-9c5a-c170187d6504
//system/sharding/products/14/c6f10055-6eae-4999-9d7b-1efd79c5bcf4
//system/sharding/products/14/RememberEntitiesStore
//system/sharding/products/16
//system/sharding/products/16/263ab2dc-7106-4023-8bdb-887019af5c09
//system/sharding/products/16/567bc4d9-19f9-463d-be72-7b396ed913cb
//system/sharding/products/16/RememberEntitiesStore
//system/sharding/products/18
//system/sharding/products/18/5040b243-7d97-4ccb-b789-d8fae36f16d1
//system/sharding/products/18/a6f0a8bd-1187-4863-be15-27556fbb79de
//system/sharding/products/18/fe0731aa-b39b-4769-9805-4abe8bd3b9b3
//system/sharding/products/18/RememberEntitiesStore
//system/sharding/products/20
//system/sharding/products/20/b8662e65-be6b-4cfa-9b72-0ccab323552d
//system/sharding/products/20/cf5d0c7b-829e-4695-8c7c-2f96a15180fa
//system/sharding/products/20/RememberEntitiesStore
//system/sharding/products/22
//system/sharding/products/22/78f210f1-7658-4714-a971-f6bce6a7c342
//system/sharding/products/22/f83d5a6d-486d-4304-967c-757b0b4a93d8
//system/sharding/products/22/RememberEntitiesStore
//system/sharding/products/24
//system/sharding/products/24/cd1ae778-254c-4fdd-91da-4394bf6cb032
//system/sharding/products/24/RememberEntitiesStore
//system/sharding/products/25
//system/sharding/products/25/3c7ec36c-e9fd-45b2-bc33-060232f2cadd
//system/sharding/products/25/3f5c54d3-c559-410d-b500-435424210194
//system/sharding/products/25/46eca73b-319e-42e0-9c89-50a3b33b82e9
//system/sharding/products/25/RememberEntitiesStore
//system/sharding/products/28
//system/sharding/products/28/4bc6dc56-b1ff-4187-a48d-24adb9b23de5
//system/sharding/products/28/RememberEntitiesStore
//system/sharding/products/29
//system/sharding/products/29/f4eb1f6f-b692-4b05-aff6-f671d6b3419f
//system/sharding/products/29/RememberEntitiesStore
//system/sharding/products/3
//system/sharding/products/3/27dc2b45-cc35-48e7-baad-806c554797c1
//system/sharding/products/3/656bd076-bf7d-4fbb-ae5f-1a245598a9b5
//system/sharding/products/3/RememberEntitiesStore
//system/sharding/products/33
//system/sharding/products/33/51eec4bc-c34b-40ad-acfd-00f7aae80fad
//system/sharding/products/33/5d18fb53-3966-4250-a264-8eb286e9a4e0
//system/sharding/products/33/RememberEntitiesStore
//system/sharding/products/35
//system/sharding/products/35/2d7ad2fa-dd02-47d4-8da8-def8a7cb4b4b
//system/sharding/products/35/8f5d2627-6fde-41ab-9c47-f9285d2e3087
//system/sharding/products/35/RememberEntitiesStore
//system/sharding/products/36
//system/sharding/products/36/536cb1c5-7734-4503-9870-22e2352d7d51
//system/sharding/products/36/RememberEntitiesStore
//system/sharding/products/38
//system/sharding/products/38/3607c89a-e71e-41c4-aee9-a1c665767bf4
//system/sharding/products/38/8381f84d-9f7e-4963-90de-11df8f4ddb50
//system/sharding/products/38/RememberEntitiesStore
//system/sharding/products/39
//system/sharding/products/39/561f10f8-ee9b-4798-9248-99b2a3971285
//system/sharding/products/39/RememberEntitiesStore
//system/sharding/products/41
//system/sharding/products/41/172bc0dd-bcdf-4109-85b5-525b0fb7a01e
//system/sharding/products/41/9bfa83d4-64aa-4367-a747-9ba3b107ef20
//system/sharding/products/41/9e6133cc-b619-4ca5-9166-7f84f03dba52
//system/sharding/products/41/RememberEntitiesStore
//system/sharding/products/45
//system/sharding/products/45/08dd424f-ed76-424a-88ed-5c70c720a6b6
//system/sharding/products/45/RememberEntitiesStore
//system/sharding/products/46
//system/sharding/products/46/ac68d893-cf23-40d5-8113-f97a9b7c9193
//system/sharding/products/46/RememberEntitiesStore
//system/sharding/products/48
//system/sharding/products/48/0f151372-581f-4f7f-bac1-d1dffdd3715e
//system/sharding/products/48/244d185e-92a6-4cad-84da-a4c24a125e2b
//system/sharding/products/48/RememberEntitiesStore
//system/sharding/products/49
//system/sharding/products/49/45524a7d-cb70-4e32-9bd3-e58afa6fb129
//system/sharding/products/49/7bd6e6b0-3d31-4ad9-a4ff-547cfd712aab
//system/sharding/products/49/d512cbc1-e2d2-40c3-a100-9ed8a3728d22
//system/sharding/products/49/RememberEntitiesStore
//system/sharding/products/5
//system/sharding/products/5/242f25b6-15c6-4671-bbd3-91e3ea02ab8d
//system/sharding/products/5/52d54b16-b2f8-4503-a6cd-88f2e3fa29c0
//system/sharding/products/5/c1d62e6c-9662-41a2-9fed-f17ae8c3dd2f
//system/sharding/products/5/RememberEntitiesStore
//system/sharding/products/6
//system/sharding/products/6/7f9d320d-4d6a-47f9-9df7-c84d5a2658e2
//system/sharding/products/6/RememberEntitiesStore
//system/sharding/products/7
//system/sharding/products/7/61eb3e6c-7a44-4ef3-8ddf-9adfa341f0b4
//system/sharding/products/7/63bfdc42-3c19-45b0-9e5e-cfa966eb732f
//system/sharding/products/7/RememberEntitiesStore
//system/sharding/products/9
//system/sharding/products/9/e8af4de6-053b-40ad-b303-204c4ec7e395
//system/sharding/products/9/RememberEntitiesStore
//system/sharding/productsCoordinator
//system/sharding/productsCoordinator/$a
//system/sharding/productsCoordinator/singleton
//system/sharding/productsCoordinator/singleton/coordinator
//system/sharding/productsCoordinator/singleton/coordinator/RememberEntitiesStore
//system/transports
//system/transports/akkaprotocolmanager.tcp.0
//system/transports/akkaprotocolmanager.tcp.0/akkaProtocol-tcp%3A%2F%2FSqlSharding%40localhost%3A7918-1
//user
//user/$a
//user/$b
//user/pbm-local-clients
//user/pbm-uptime
//user/petabridge.cmd
//user/petabridge.cmd/127.0.0.1%3A54340
//user/petabridge.cmd/127.0.0.1%3A54340/handler
//user/petabridge.cmd/127.0.0.1%3A54340/handler/actor
//user/petabridge.cmd/127.0.0.1%3A54340/handler/actor/handler-0
//user/petabridge.cmd/127.0.0.1%3A54340/handler/actor/handler-0/$a
//user/petabridge.cmd-log-memorizer
//user/product-index
//user/product-index/$a
//user/product-index/product-index
//user/product-proxy
The message Persistent
refers to this class:
https://github.com/akkadotnet/akka.net/blob/eba7bf50d4766ca7160edad6af18bdaf4970fbc7/src/core/Akka.Persistence/Persistent.cs#L297
Version Information Version of Akka.NET? v1.5.0-alpha1 Which Akka.NET Modules? Akka.Cluster.Sharding
Describe the bug
It looks like the Akka.Cluster.Sharding
PersistentCoordinator
is not able to correctly run its shard migration, as we see error messages like this at startup:And in addition to that, the Akka.Persistence.SqlServer journal returns no results for
SM
- indicating that the migration of previous remember-entities data was executed:To Reproduce
Reproduction instructions and data available here: https://github.com/petabridge/akkadotnet-code-samples/pull/112
Expected behavior
All remembered-entities should start automatically.
Actual behavior
None do.
Additional context
I used a
pbm actor hierarchy -s akka://SqlSharding/ -d 6
command to do a dump of the entire actor hierarchy after the sharding system started correctly and here's what I found:This verifies that none of the remembered entities or shards started correctly.