search

datastax / dsbulk

DataStax Bulk Loader (DSBulk) is an open-source, Apache-licensed, unified tool for loading into and unloading from Apache Cassandra(R), DataStax Astra and DataStax Enterprise (DSE)
Apache License 2.0
85 stars 30 forks source link

endless loop on initial connection, when connecting to Astra and one of the endpoints is mark as removed #454

Open pidiaquez opened 2 years ago

pidiaquez commented 2 years ago

dsbulk 1.10.0 will fall into an endless loop when one of the contact points is not longer valid

command line:

dsbulk count -verbosity max -reportRate 1 -k keyspace -t table -b "/root/secure-connecttest-02.zip" -u username

IP 20.1.1.1 represents Public IP reachable

at the time of the event one Contact point was valid and the other Contact point was invalid.

only 2 contact points ( public IPs)

2022-09-15 15:16:32 DEBUG driver-io-4     c.d.o.d.i.core.channel.ProtocolInitHandler    [driver|control|connecting...] Starting channel initialization
2022-09-15 15:16:32 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|connecting...] Writing init query STARTUP on stream id 0
2022-09-15 15:16:32 DEBUG driver-io-4     io.netty.handler.ssl.SslHandler               [id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] HANDSHAKEN: protocol:TLSv1.2 cipher suite:SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384
2022-09-15 15:16:32 DEBUG driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Got last response on in-flight stream id 0, completing and releasing
2022-09-15 15:16:32 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Releasing stream id 0
2022-09-15 15:16:32 DEBUG driver-io-4     c.d.o.d.i.core.channel.ProtocolInitHandler    [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] step STARTUP received response opcode=AUTHENTICATE
2022-09-15 15:16:32 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Writing init query AUTH_RESPONSE on stream id 0
2022-09-15 15:16:33 DEBUG driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Got last response on in-flight stream id 0, completing and releasing
2022-09-15 15:16:33 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Releasing stream id 0
2022-09-15 15:16:33 DEBUG driver-io-4     c.d.o.d.i.core.channel.ProtocolInitHandler    [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] step AUTH_RESPONSE received response opcode=AUTH_SUCCESS
2022-09-15 15:16:33 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Writing init query GET_CLUSTER_NAME on stream id 0
2022-09-15 15:16:34 DEBUG driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Got last response on in-flight stream id 0, completing and releasing
2022-09-15 15:16:34 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Releasing stream id 0
2022-09-15 15:16:34 DEBUG driver-io-4     c.d.o.d.i.core.channel.ProtocolInitHandler    [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] step GET_CLUSTER_NAME received response opcode=RESULT
2022-09-15 15:16:34 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Writing init query REGISTER on stream id 0
2022-09-15 15:16:34 DEBUG driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Got last response on in-flight stream id 0, completing and releasing
2022-09-15 15:16:34 TRACE driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Releasing stream id 0
2022-09-15 15:16:34 DEBUG driver-io-4     c.d.o.d.i.core.channel.ProtocolInitHandler    [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] step REGISTER received response opcode=READY
2022-09-15 15:16:34 DEBUG driver-admin-1  c.d.o.d.i.core.control.ControlConnection      [driver] New channel opened ([id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044]) but node was removed or forced down, closing and trying next node
2022-09-15 15:16:34 DEBUG driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Received graceful close request
2022-09-15 15:16:34 DEBUG driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] No pending queries, completing graceful shutdown now
2022-09-15 15:16:34 DEBUG driver-io-4     c.d.o.d.internal.core.channel.InFlightHandler [driver|control|id: 0x5d2aec28, L:/10.0.0.1:38992 - R:dbuuid-redacted-eastus.db.astra.datastax.com/20.1.1.1:30044] Received forceful close request, aborting pending queries
2022-09-15 15:16:34 DEBUG driver-admin-1  c.d.o.d.i.core.control.ControlConnection      [driver] Trying to establish a connection to Node(endPoint=dbuuid-redacted-eastus.db.astra.datastax.com:30044:00000000-0000-3000-8000-020051211183, hostId=null, hashCode=b9e2d7bb)

┆Issue is synchronized with this Jira Task by Unito