search

akka / akka

A platform to build and run apps that are elastic, agile, and resilient. SDK, libraries, and hosted environments.
https://doc.akka.io
Other
13.05k stars 3.59k forks source link

failed: JoinConfigCompatCheckerSpec #30843

Open patriknw opened 3 years ago

patriknw commented 3 years ago

https://github.com/akka/akka/runs/4062612167?check_suite_focus=true#step:5:21377

[info] - must NOT be allowed to re-join a cluster when one of its required properties are not available on cluster side *** FAILED *** (51 seconds, 61 milliseconds)
21378
[info]   java.lang.AssertionError: assertion failed: timeout 49098088170 nanoseconds expired: awaiting node [akka://JoinConfigCompatCheckerSpec@10.1.0.220:34849] to be 'Removed'. Current status: [Up]
21379
[info]   at scala.Predef$.assert(Predef.scala:223)
21380
[info]   at akka.testkit.TestKitBase.poll$1(TestKit.scala:300)
21381
[info]   at akka.testkit.TestKitBase.awaitCond(TestKit.scala:306)
21382
[info]   at akka.testkit.TestKitBase.awaitCond$(TestKit.scala:289)
21383
[info]   at akka.testkit.TestKit.awaitCond(TestKit.scala:973)
21384
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.quitAndRestart(ClusterTestKit.scala:123)
21385
[info]   at akka.cluster.JoinConfigCompatCheckerSpec.$anonfun$new$30(JoinConfigCompatCheckerSpec.scala:400)
21386
[info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
21387
[info]   at akka.testkit.TestKitBase.within(TestKit.scala:418)
21388
[info]   at akka.testkit.TestKitBase.within$(TestKit.scala:406)
21389
[info]   at akka.testkit.TestKit.within(TestKit.scala:973)
21390
[info]   at akka.testkit.TestKitBase.within(TestKit.scala:433)
21391
[info]   at akka.testkit.TestKitBase.within$(TestKit.scala:433)
21392
[info]   at akka.testkit.TestKit.within(TestKit.scala:973)
21393
[info]   at akka.cluster.JoinConfigCompatCheckerSpec.$anonfun$new$29(JoinConfigCompatCheckerSpec.scala:396)
patriknw commented 2 years ago

https://github.com/akka/akka/runs/4402427160?check_suite_focus=true#step:5:21684

octonato commented 2 years ago

This is back: https://github.com/akka/akka/runs/4915607313?check_suite_focus=true#step:5:1240

Build from 9 days ago

patriknw commented 2 years ago

https://github.com/akka/akka/runs/5712840176?check_suite_focus=true#step:5:1380

[INFO] [03/28/2022 00:17:13.936] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Remoting] Starting remoting
1265
[INFO] [03/28/2022 00:17:13.939] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Remoting] Remoting started; listening on addresses :[akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]
1266
[INFO] [03/28/2022 00:17:13.939] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Remoting] Remoting now listens on addresses: [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]
1267
[INFO] [03/28/2022 00:17:13.940] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Starting up, Akka version [2.6.19+11-8b42165c-SNAPSHOT] ...
1268
[INFO] [03/28/2022 00:17:13.942] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Registered cluster JMX MBean [akka:type=Cluster,port=43485]
1269
[INFO] [03/28/2022 00:17:13.942] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Started up successfully
1270
[INFO] [03/28/2022 00:17:13.947] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-5] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing
1271
[INFO] [03/28/2022 00:17:13.947] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-5] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining
1272
[INFO] [03/28/2022 00:17:13.974] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Remoting] Starting remoting
1273
[INFO] [03/28/2022 00:17:13.991] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Remoting] Remoting started; listening on addresses :[akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581]
1274
[INFO] [03/28/2022 00:17:13.991] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Remoting] Remoting now listens on addresses: [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581]
1275
[INFO] [03/28/2022 00:17:13.991] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Starting up, Akka version [2.6.19+11-8b42165c-SNAPSHOT] ...
1276
[INFO] [03/28/2022 00:17:13.995] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Registered cluster JMX MBean [akka:type=Cluster,port=44581]
1277
[INFO] [03/28/2022 00:17:13.995] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Started up successfully
1278
[INFO] [03/28/2022 00:17:14.004] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing
1279
[INFO] [03/28/2022 00:17:14.004] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining
1280
[INFO] [akkaMemberChanged][03/28/2022 00:17:14.020] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] is JOINING itself (with roles [dc-default], version [0.0.0]) and forming new cluster
1281
[INFO] [03/28/2022 00:17:14.020] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - is the new leader among reachable nodes (more leaders may exist)
1282
[INFO] [akkaMemberChanged][03/28/2022 00:17:14.020] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Leader is moving node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] to [Up]
1283
[INFO] [03/28/2022 00:17:14.030] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-5] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Received InitJoin message from [Actor[akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/cluster/core/daemon/joinSeedNodeProcess-1#-1254692192]] to [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]
1284
[INFO] [03/28/2022 00:17:14.030] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-5] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Sending InitJoinAck message from node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] to [Actor[akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/cluster/core/daemon/joinSeedNodeProcess-1#-1254692192]] (version [2.6.19+11-8b42165c-SNAPSHOT])
1285
[INFO] [03/28/2022 00:17:14.041] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-4] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Received InitJoinAck message from [Actor[akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485/system/cluster/core/daemon#-812059085]] to [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581]
1286
[INFO] [akkaMemberChanged][03/28/2022 00:17:14.046] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-5] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] is JOINING, roles [dc-default], version [0.0.0]
1287
[INFO] [03/28/2022 00:17:14.048] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Welcome from [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]
1288
[INFO] [akkaMemberChanged][03/28/2022 00:17:14.961] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Leader is moving node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] to [Up]
1289
[INFO] [akkaMemberChanged][03/28/2022 00:17:15.028] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Marked address [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] as [Leaving]
1290
[INFO] [03/28/2022 00:17:15.981] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-3] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Exiting (leader), starting coordinated shutdown
1291
[INFO] [03/28/2022 00:17:15.981] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-3] [CoordinatedShutdown(akka://JoinConfigCompatCheckerSpec)] Running CoordinatedShutdown with reason [ClusterLeavingReason]
1292
[INFO] [akkaMemberChanged][03/28/2022 00:17:15.981] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-3] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Leader is moving node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] to [Exiting]
1293
[INFO] [03/28/2022 00:17:15.982] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-4] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Exiting completed
1294
[INFO] [03/28/2022 00:17:15.982] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-4] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Shutting down...
1295
[INFO] [03/28/2022 00:17:15.982] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-4] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] - Successfully shut down
1296
[INFO] [03/28/2022 00:17:15.983] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-8] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485/system/remoting-terminator] Shutting down remote daemon.
1297
[INFO] [03/28/2022 00:17:15.983] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-8] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports.
1298
[WARN] [03/28/2022 00:17:15.987] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-8] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Disassociated] 
1299
[INFO] [03/28/2022 00:17:15.988] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [Remoting] Remoting shut down
1300
[INFO] [03/28/2022 00:17:15.988] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-8] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485/system/remoting-terminator] Remoting shut down.
1301
[INFO] [akkaDeadLetter][03/28/2022 00:17:16.036] [JoinConfigCompatCheckerSpec-akka.actor.default-dispatcher-17] [akka://JoinConfigCompatCheckerSpec/deadLetters] Message [akka.cluster.GossipStatus] from Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon#1218978067] to Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
1302
[INFO] [akkaDeadLetter][03/28/2022 00:17:17.056] [JoinConfigCompatCheckerSpec-akka.actor.default-dispatcher-17] [akka://JoinConfigCompatCheckerSpec/deadLetters] Message [akka.cluster.GossipStatus] from Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon#1218978067] to Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
1303
[INFO] [akkaDeadLetter][03/28/2022 00:17:18.076] [JoinConfigCompatCheckerSpec-akka.actor.default-dispatcher-17] [akka://JoinConfigCompatCheckerSpec/deadLetters] Message [akka.cluster.GossipStatus] from Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon#1218978067] to Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
1304
[INFO] [akkaDeadLetter][03/28/2022 00:17:19.056] [JoinConfigCompatCheckerSpec-akka.actor.default-dispatcher-17] [akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon/joinSeedNodeProcess-1] Message [akka.actor.ReceiveTimeout$] to Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon/joinSeedNodeProcess-1#-1254692192] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon/joinSeedNodeProcess-1#-1254692192] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
1305
[INFO] [akkaDeadLetter][03/28/2022 00:17:19.087] [JoinConfigCompatCheckerSpec-akka.actor.default-dispatcher-17] [akka://JoinConfigCompatCheckerSpec/deadLetters] Message [akka.cluster.GossipStatus] from Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon#1218978067] to Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
1306
[INFO] [akkaDeadLetter][03/28/2022 00:17:20.097] [JoinConfigCompatCheckerSpec-akka.actor.default-dispatcher-17] [akka://JoinConfigCompatCheckerSpec/deadLetters] Message [akka.cluster.GossipStatus] from Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon#1218978067] to Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
1307
[WARN] [03/28/2022 00:17:21.110] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1308
[WARN] [akkaUnreachable][03/28/2022 00:17:21.111] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-5] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Marking node as UNREACHABLE [Member(akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485, Leaving)].
1309
[INFO] [03/28/2022 00:17:21.112] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-5] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - is the new leader among reachable nodes (more leaders may exist)
1310
[WARN] [03/28/2022 00:17:21.113] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1311
[INFO] [akkaDeadLetter][03/28/2022 00:17:21.113] [JoinConfigCompatCheckerSpec-akka.actor.default-dispatcher-6] [akka://JoinConfigCompatCheckerSpec/deadLetters] Message [akka.cluster.GossipStatus] from Actor[akka://JoinConfigCompatCheckerSpec/system/cluster/core/daemon#1218978067] to Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://JoinConfigCompatCheckerSpec/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
1312
[WARN] [03/28/2022 00:17:26.289] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1313
[WARN] [03/28/2022 00:17:26.289] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1314
[WARN] [03/28/2022 00:17:31.378] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1315
[WARN] [03/28/2022 00:17:31.378] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1316
[WARN] [03/28/2022 00:17:36.478] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1317
[WARN] [03/28/2022 00:17:36.479] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1318
[INFO] [akkaClusterLeaderIncapacitated][03/28/2022 00:17:40.427] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Leader can currently not perform its duties, reachability status: [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581 -> akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485: Unreachable [Unreachable] (1)], member status: [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485 Leaving seen=false, akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581 Up seen=true]
1319
[WARN] [03/28/2022 00:17:41.578] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1320
[WARN] [03/28/2022 00:17:41.580] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1321
[WARN] [03/28/2022 00:17:46.678] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1322
[WARN] [03/28/2022 00:17:46.679] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1323
[WARN] [03/28/2022 00:17:51.780] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1324
[WARN] [03/28/2022 00:17:51.781] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1325
[WARN] [03/28/2022 00:17:56.878] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1326
[WARN] [03/28/2022 00:17:56.880] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1327
[114.236s][info][gc,start      ] GC(9) Pause Young (Concurrent Start) (Metadata GC Threshold)
1328
[114.236s][info][gc,task       ] GC(9) Using 2 workers of 2 for evacuation
1329
[114.256s][info][gc,phases     ] GC(9)   Pre Evacuate Collection Set: 0.3ms
1330
[114.256s][info][gc,phases     ] GC(9)   Evacuate Collection Set: 17.4ms
1331
[114.256s][info][gc,phases     ] GC(9)   Post Evacuate Collection Set: 1.4ms
1332
[114.256s][info][gc,phases     ] GC(9)   Other: 0.8ms
1333
[114.256s][info][gc,heap       ] GC(9) Eden regions: 175->0(320)
1334
[114.256s][info][gc,heap       ] GC(9) Survivor regions: 12->12(231)
1335
[114.256s][info][gc,heap       ] GC(9) Old regions: 8->8
1336
[114.256s][info][gc,heap       ] GC(9) Humongous regions: 0->0
1337
[114.256s][info][gc,metaspace  ] GC(9) Metaspace: 57564K->57564K(1101824K)
1338
[114.256s][info][gc            ] GC(9) Pause Young (Concurrent Start) (Metadata GC Threshold) 193M->18M(3072M) 19.872ms
1339
[114.256s][info][gc,cpu        ] GC(9) User=0.03s Sys=0.00s Real=0.02s
1340
[114.256s][info][gc            ] GC(10) Concurrent Cycle
1341
[114.256s][info][gc,marking    ] GC(10) Concurrent Clear Claimed Marks
1342
[114.256s][info][gc,marking    ] GC(10) Concurrent Clear Claimed Marks 0.233ms
1343
[114.256s][info][gc,marking    ] GC(10) Concurrent Scan Root Regions
1344
[114.266s][info][gc,marking    ] GC(10) Concurrent Scan Root Regions 9.508ms
1345
[114.266s][info][gc,marking    ] GC(10) Concurrent Mark (114.266s)
1346
[114.266s][info][gc,marking    ] GC(10) Concurrent Mark From Roots
1347
[114.266s][info][gc,task       ] GC(10) Using 1 workers of 1 for marking
1348
[114.278s][info][gc,marking    ] GC(10) Concurrent Mark From Roots 11.925ms
1349
[114.278s][info][gc,marking    ] GC(10) Concurrent Preclean
1350
[114.278s][info][gc,marking    ] GC(10) Concurrent Preclean 0.053ms
1351
[114.278s][info][gc,marking    ] GC(10) Concurrent Mark (114.266s, 114.278s) 12.161ms
1352
[114.278s][info][gc,start      ] GC(10) Pause Remark
1353
[114.291s][info][gc,stringtable] GC(10) Cleaned string and symbol table, strings: 14733 processed, 10 removed, symbols: 128994 processed, 22 removed
1354
[114.292s][info][gc            ] GC(10) Pause Remark 18M->18M(3072M) 13.995ms
1355
[114.292s][info][gc,cpu        ] GC(10) User=0.02s Sys=0.00s Real=0.02s
1356
[114.292s][info][gc,marking    ] GC(10) Concurrent Rebuild Remembered Sets
1357
[114.297s][info][gc,marking    ] GC(10) Concurrent Rebuild Remembered Sets 4.730ms
1358
[114.297s][info][gc,start      ] GC(10) Pause Cleanup
1359
[114.298s][info][gc            ] GC(10) Pause Cleanup 18M->18M(3072M) 0.571ms
1360
[114.298s][info][gc,cpu        ] GC(10) User=0.00s Sys=0.00s Real=0.00s
1361
[114.298s][info][gc,marking    ] GC(10) Concurrent Cleanup for Next Mark
1362
[114.307s][info][gc,marking    ] GC(10) Concurrent Cleanup for Next Mark 8.854ms
1363
[114.307s][info][gc            ] GC(10) Concurrent Cycle 50.985ms
1364
[WARN] [03/28/2022 00:18:01.978] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1365
[WARN] [03/28/2022 00:18:01.979] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-8] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1366
[INFO] [03/28/2022 00:18:03.991] [pool-1-thread-1-ScalaTest-running-JoinConfigCompatCheckerSpec] [CoordinatedShutdown(akka://JoinConfigCompatCheckerSpec)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
1367
[INFO] [akkaMemberChanged][03/28/2022 00:18:03.992] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-4] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Marked address [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] as [Leaving]
1368
[WARN] [03/28/2022 00:18:07.068] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1369
[WARN] [03/28/2022 00:18:07.069] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1370
[WARN] [03/28/2022 00:18:12.169] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-8] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FJoinConfigCompatCheckerSpec%4010.1.0.126%3A43485-0] Association with remote system [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485]] Caused by: [java.net.ConnectException: Connection refused: /10.1.0.126:43485]
1371
[WARN] [03/28/2022 00:18:12.169] [New I/O boss #219] [NettyTransport(akka://JoinConfigCompatCheckerSpec)] Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.1.0.126:43485
1372
[WARN] [03/28/2022 00:18:14.006] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [CoordinatedShutdown(akka://JoinConfigCompatCheckerSpec)] Coordinated shutdown phase [cluster-exiting] timed out after 10000 milliseconds
1373
[INFO] [03/28/2022 00:18:14.006] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Exiting completed
1374
[INFO] [03/28/2022 00:18:14.006] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Shutting down...
1375
[INFO] [03/28/2022 00:18:14.006] [JoinConfigCompatCheckerSpec-akka.actor.internal-dispatcher-2] [Cluster(akka://JoinConfigCompatCheckerSpec)] Cluster Node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581] - Successfully shut down
1376
[INFO] [03/28/2022 00:18:14.008] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/remoting-terminator] Shutting down remote daemon.
1377
[INFO] [03/28/2022 00:18:14.008] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports.
1378
[INFO] [03/28/2022 00:18:14.016] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [Remoting] Remoting shut down
1379
[INFO] [03/28/2022 00:18:14.017] [JoinConfigCompatCheckerSpec-akka.remote.default-remote-dispatcher-7] [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581/system/remoting-terminator] Remoting shut down.
1380
[info] - must be allowed to re-join a cluster when its configuration is incompatible but it's configured to NOT enforce it *** FAILED *** (1 minute)
1381
[info]   java.lang.AssertionError: awaiting node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485] to be removed, still member in [akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581]: CurrentClusterState(TreeSet(Member(akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485, Leaving), Member(akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581, Up)), Set(Member(akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:43485, Leaving)), Set(akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581), Some(akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581), Map(dc-default -> Some(akka.tcp://JoinConfigCompatCheckerSpec@10.1.0.126:44581)))
1382
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.$anonfun$quitAndRestart$4(ClusterTestKit.scala:128)
1383
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.$anonfun$quitAndRestart$4$adapted(ClusterTestKit.scala:122)
1384
[info]   at scala.collection.immutable.List.foreach(List.scala:333)
1385
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.$anonfun$quitAndRestart$3(ClusterTestKit.scala:122)
1386
[info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
1387
[info]   at akka.testkit.TestKitBase.poll$2(TestKit.scala:332)
1388
[info]   at akka.testkit.TestKitBase.awaitAssert(TestKit.scala:349)
1389
[info]   at akka.testkit.TestKitBase.awaitAssert$(TestKit.scala:321)
1390
[info]   at akka.testkit.TestKit.awaitAssert(TestKit.scala:973)
1391
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.quitAndRestart(ClusterTestKit.scala:121)
1392
[info]   at akka.cluster.JoinConfigCompatCheckerSpec.$anonfun$new$42(JoinConfigCompatCheckerSpec.scala:539)
1393
[info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
1394
[info]   at akka.testkit.TestKitBase.within(TestKit.scala:418)
1395
[info]   at akka.testkit.TestKitBase.within$(TestKit.scala:406)
1396
[info]   at akka.testkit.TestKit.within(TestKit.scala:973)
1397
[info]   at akka.testkit.TestKitBase.within(TestKit.scala:433)
1398
[info]   at akka.testkit.TestKitBase.within$(TestKit.scala:433)
1399
[info]   at akka.testkit.TestKit.within(TestKit.scala:973)
1400
[info]   at akka.cluster.JoinConfigCompatCheckerSpec.$anonfun$new$41(JoinConfigCompatCheckerSpec.scala:534)
pvlugter commented 2 years ago

https://github.com/akka/akka/runs/5808939713?check_suite_focus=true#step:5:1341

octonato commented 2 years ago

https://github.com/akka/akka/runs/6136665666?check_suite_focus=true#step:5:1219

- must NOT be allowed to re-join a cluster when one of the cluster required properties are not available on the joining side *** FAILED *** (1 minute)
[info]   java.lang.AssertionError: awaiting node [akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:34331] to be removed, still member in [akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:43363]: CurrentClusterState(TreeSet(Member(akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:34331, Leaving), Member(akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:43363, Up)), Set(Member(akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:34331, Leaving)), Set(akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:43363), Some(akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:43363), Map(dc-default -> Some(akka.tcp://JoinConfigCompatCheckerSpec@10.1.1.219:43363)))
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.$anonfun$quitAndRestart$4(ClusterTestKit.scala:128)
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.$anonfun$quitAndRestart$4$adapted(ClusterTestKit.scala:122)
[info]   at scala.collection.immutable.List.foreach(List.scala:333)
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.$anonfun$quitAndRestart$3(ClusterTestKit.scala:122)
[info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[info]   at akka.testkit.TestKitBase.poll$2(TestKit.scala:332)
[info]   at akka.testkit.TestKitBase.awaitAssert(TestKit.scala:349)
[info]   at akka.testkit.TestKitBase.awaitAssert$(TestKit.scala:321)
[info]   at akka.testkit.TestKit.awaitAssert(TestKit.scala:973)
[info]   at akka.cluster.ClusterTestKit$ClusterTestUtil.quitAndRestart(ClusterTestKit.scala:121)
[info]   at akka.cluster.JoinConfigCompatCheckerSpec.$anonfun$new$34(JoinConfigCompatCheckerSpec.scala:445)
[info]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[info]   at akka.testkit.TestKitBase.within(TestKit.scala:418)
[info]   at akka.testkit.TestKitBase.within$(TestKit.scala:406)
[info]   at akka.testkit.TestKit.within(TestKit.scala:973)
[info]   at akka.testkit.TestKitBase.within(TestKit.scala:433)
[info]   at akka.testkit.TestKitBase.within$(TestKit.scala:433)
[info]   at akka.testkit.TestKit.within(TestKit.scala:973)
pvlugter commented 2 years ago

https://github.com/akka/akka/runs/7033821559?check_suite_focus=true#step:5:1391

johanandren commented 2 years ago

https://github.com/akka/akka/runs/7103285935?check_suite_focus=true#step:5:1268

patriknw commented 2 years ago

https://github.com/akka/akka/runs/8147381006?check_suite_focus=true#step:5:1394

pvlugter commented 2 years ago

https://github.com/akka/akka/actions/runs/3147770388/jobs/5117595589#step:5:1107

patriknw commented 2 years ago

https://github.com/akka/akka/actions/runs/3432880859/jobs/5722605777#step:5:1401

patriknw commented 1 year ago

https://github.com/akka/akka/actions/runs/3484194539/jobs/5828480347#step:5:1005

johanandren commented 1 year ago

https://github.com/akka/akka/actions/runs/3560604983/jobs/5980744741#step:5:1360