search

scylladb / scylla-migrator

Migrate data extract using Spark to Scylla, normally from Cassandra
Apache License 2.0
54 stars 34 forks source link

spark connect to Cassandra db is get a network reset #98

Open xzhaous opened 1 year ago

xzhaous commented 1 year ago

docker-compose exec spark-master /spark/bin/spark-submit --class com.scylladb.migrator.Migrator \

--master spark://spark-master:7077 \ --conf spark.driver.host=spark-master \ --conf spark.scylla.config=/app/config.yaml \ --conf spark.files=/app/secure-connect-clio-ecu1.zip \ --conf spark.cassandra.connection.config.cloud.path=secure-connect-clio-ecu1.zip \ --conf spark.cassandra.auth.username=BUWFQGmIjlOGvaxRbpwacpsk \ --conf spark.cassandra.auth.password=XXXXXXXXXXXXXXXXXX /jars/scylla-migrator-assembly-0.0.1.jar 23/07/13 20:29:33 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 23/07/13 20:29:34 INFO SparkContext: Running Spark version 2.4.4 23/07/13 20:29:34 INFO SparkContext: Submitted application: scylla-migrator 23/07/13 20:29:34 INFO SecurityManager: Changing view acls to: root 23/07/13 20:29:34 INFO SecurityManager: Changing modify acls to: root 23/07/13 20:29:34 INFO SecurityManager: Changing view acls groups to: 23/07/13 20:29:34 INFO SecurityManager: Changing modify acls groups to: 23/07/13 20:29:34 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set() 23/07/13 20:29:34 INFO Utils: Successfully started service 'sparkDriver' on port 43083. 23/07/13 20:29:34 INFO SparkEnv: Registering MapOutputTracker 23/07/13 20:29:34 INFO SparkEnv: Registering BlockManagerMaster 23/07/13 20:29:34 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information 23/07/13 20:29:34 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up 23/07/13 20:29:34 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-bdd71853-edd9-429a-9fbf-c8418e375f31 23/07/13 20:29:34 INFO MemoryStore: MemoryStore started with capacity 366.3 MB 23/07/13 20:29:34 INFO SparkEnv: Registering OutputCommitCoordinator 23/07/13 20:29:35 INFO Utils: Successfully started service 'SparkUI' on port 4040. 23/07/13 20:29:35 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://spark-master:4040 23/07/13 20:29:35 INFO SparkContext: Added JAR file:/jars/scylla-migrator-assembly-0.0.1.jar at spark://spark-master:43083/jars/scylla-migrator-assembly-0.0.1.jar with timestamp 1689280175223 23/07/13 20:29:35 INFO SparkContext: Added file file:///app/secure-connect-clio-ecu1.zip at spark://spark-master:43083/files/secure-connect-clio-ecu1.zip with timestamp 1689280175242 23/07/13 20:29:35 INFO Utils: Copying /app/secure-connect-clio-ecu1.zip to /tmp/spark-953942cb-2243-4f11-94f8-cb27efc933bc/userFiles-efc7e339-b49b-427f-b192-c7258a1e6e5f/secure-connect-clio-ecu1.zip 23/07/13 20:29:35 INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark://spark-master:7077... 23/07/13 20:29:35 INFO TransportClientFactory: Successfully created connection to spark-master/192.168.0.3:7077 after 33 ms (0 ms spent in bootstraps) 23/07/13 20:29:35 INFO StandaloneSchedulerBackend: Connected to Spark cluster with app ID app-20230713202935-0001 23/07/13 20:29:35 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20230713202935-0001/0 on worker-20230713195730-192.168.0.5-34811 (192.168.0.5:34811) with 3 core(s) 23/07/13 20:29:35 INFO StandaloneSchedulerBackend: Granted executor ID app-20230713202935-0001/0 on hostPort 192.168.0.5:34811 with 3 core(s), 1024.0 MB RAM 23/07/13 20:29:35 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 46693. 23/07/13 20:29:35 INFO NettyBlockTransferService: Server created on spark-master:46693 23/07/13 20:29:35 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 23/07/13 20:29:35 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20230713202935-0001/0 is now RUNNING 23/07/13 20:29:35 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, spark-master, 46693, None) 23/07/13 20:29:35 INFO BlockManagerMasterEndpoint: Registering block manager spark-master:46693 with 366.3 MB RAM, BlockManagerId(driver, spark-master, 46693, None) 23/07/13 20:29:35 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, spark-master, 46693, None) 23/07/13 20:29:35 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, spark-master, 46693, None) 23/07/13 20:29:35 INFO StandaloneSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0 23/07/13 20:29:37 INFO migrator: Loaded config: MigratorConfig(Cassandra(4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com,30042,None,Some(Credentials(XXXXXXXXXXXXXXXXX),Scylla(node-0.aws-eu-central-1.1eafc16798f1518ee554.clusters.scylla.cloud,9042,None,Some(Credentials(scylla,XXXXXXXX)),None,clio_fh,ledgers,Some(16),false,None,None,LOCAL_QUORUM),List(),Savepoints(300,/app/savepoints),Set(),Validation(true,60000,1000,100,0.001,0)) 23/07/13 20:29:37 INFO Cassandra: Using consistencyLevel [LOCAL_QUORUM] for SOURCE based on source config [LOCAL_QUORUM] 23/07/13 20:29:40 WARN ControlConnection: [s0] Error connecting to Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/18.196.17.10:30042, hostId=null, hashCode=355b44a1), trying next node (ConnectionInitException: [s0|control|id: 0xa8c9360f, L:/192.168.0.3:42542 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/18.196.17.10:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)) 23/07/13 20:29:40 WARN ControlConnection: [s0] Error connecting to Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/54.93.231.213:30042, hostId=null, hashCode=6b00c60b), trying next node (ConnectionInitException: [s0|control|id: 0xc889594a, L:/192.168.0.3:44688 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/54.93.231.213:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)) 23/07/13 20:29:40 WARN ControlConnection: [s0] Error connecting to Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.76.47.214:30042, hostId=null, hashCode=9b8eb3), trying next node (ConnectionInitException: [s0|control|id: 0xa84ef53a, L:/192.168.0.3:59560 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.76.47.214:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)) 23/07/13 20:29:40 WARN ControlConnection: [s0] Error connecting to Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.74.47.48:30042, hostId=null, hashCode=1858f101), trying next node (ConnectionInitException: [s0|control|id: 0x22ee7c69, L:/192.168.0.3:46780 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.74.47.48:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)) Exception in thread "main" java.io.IOException: Failed to open native connection to Cassandra at {4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com:30042} :: Could not reach any contact point, make sure you've provided valid addresses (showing first 3 nodes, use getAllErrors() for more): Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/18.196.17.10:30042, hostId=null, hashCode=355b44a1): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xa8c9360f, L:/192.168.0.3:42542 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/18.196.17.10:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)], Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/54.93.231.213:30042, hostId=null, hashCode=6b00c60b): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xc889594a, L:/192.168.0.3:44688 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/54.93.231.213:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)], Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.76.47.214:30042, hostId=null, hashCode=9b8eb3): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xa84ef53a, L:/192.168.0.3:59560 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.76.47.214:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)] at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:181) at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:169) at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:169) at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:32) at com.datastax.spark.connector.cql.RefCountedCache.syncAcquire(RefCountedCache.scala:69) at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:57) at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:89) at com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:111) at com.scylladb.migrator.readers.Cassandra$.readDataframe(Cassandra.scala:231) at com.scylladb.migrator.Migrator$.main(Migrator.scala:47) at com.scylladb.migrator.Migrator.main(Migrator.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: com.datastax.oss.driver.api.core.AllNodesFailedException: Could not reach any contact point, make sure you've provided valid addresses (showing first 3 nodes, use getAllErrors() for more): Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/18.196.17.10:30042, hostId=null, hashCode=355b44a1): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xa8c9360f, L:/192.168.0.3:42542 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/18.196.17.10:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)], Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/54.93.231.213:30042, hostId=null, hashCode=6b00c60b): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xc889594a, L:/192.168.0.3:44688 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/54.93.231.213:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)], Node(endPoint=4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.76.47.214:30042, hostId=null, hashCode=9b8eb3): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xa84ef53a, L:/192.168.0.3:59560 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.76.47.214:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)] at com.datastax.oss.driver.api.core.AllNodesFailedException.copy(AllNodesFailedException.java:141) at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149) at com.datastax.oss.driver.api.core.session.SessionBuilder.build(SessionBuilder.java:612) at com.datastax.spark.connector.cql.DefaultConnectionFactory$.createSession(CassandraConnectionFactory.scala:141) at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:175) ... 22 more Suppressed: com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xa8c9360f, L:/192.168.0.3:42542 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/18.196.17.10:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer) at com.datastax.oss.driver.internal.core.channel.ProtocolInitHandler$InitRequest.fail(ProtocolInitHandler.java:342) at com.datastax.oss.driver.internal.core.channel.ChannelHandlerRequest.onFailure(ChannelHandlerRequest.java:104) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.fail(InFlightHandler.java:373) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:362) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:351) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.channelInactive(InFlightHandler.java:329) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:386) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:351) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818) at com.datastax.oss.driver.shaded.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at com.datastax.oss.driver.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at com.datastax.oss.driver.shaded.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at com.datastax.oss.driver.shaded.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer Suppressed: com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xc889594a, L:/192.168.0.3:44688 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/54.93.231.213:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer) at com.datastax.oss.driver.internal.core.channel.ProtocolInitHandler$InitRequest.fail(ProtocolInitHandler.java:342) at com.datastax.oss.driver.internal.core.channel.ChannelHandlerRequest.onFailure(ChannelHandlerRequest.java:104) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.fail(InFlightHandler.java:373) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:362) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:351) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.channelInactive(InFlightHandler.java:329) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:386) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:351) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818) at com.datastax.oss.driver.shaded.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at com.datastax.oss.driver.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at com.datastax.oss.driver.shaded.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at com.datastax.oss.driver.shaded.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer Suppressed: com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0xa84ef53a, L:/192.168.0.3:59560 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.76.47.214:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer) at com.datastax.oss.driver.internal.core.channel.ProtocolInitHandler$InitRequest.fail(ProtocolInitHandler.java:342) at com.datastax.oss.driver.internal.core.channel.ChannelHandlerRequest.onFailure(ChannelHandlerRequest.java:104) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.fail(InFlightHandler.java:373) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:362) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:351) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.channelInactive(InFlightHandler.java:329) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:386) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:351) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818) at com.datastax.oss.driver.shaded.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at com.datastax.oss.driver.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at com.datastax.oss.driver.shaded.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at com.datastax.oss.driver.shaded.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer Suppressed: com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0x22ee7c69, L:/192.168.0.3:46780 - R:4f2b5aa6-6f49-4677-baec-dfc9208f35e0-eu-central-1.db.astra.datastax.com/3.74.47.48:30042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer) at com.datastax.oss.driver.internal.core.channel.ProtocolInitHandler$InitRequest.fail(ProtocolInitHandler.java:342) at com.datastax.oss.driver.internal.core.channel.ChannelHandlerRequest.onFailure(ChannelHandlerRequest.java:104) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.fail(InFlightHandler.java:373) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:362) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.abortAllInFlight(InFlightHandler.java:351) at com.datastax.oss.driver.internal.core.channel.InFlightHandler.channelInactive(InFlightHandler.java:329) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:386) at com.datastax.oss.driver.shaded.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:351) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:239) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:260) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:246) at com.datastax.oss.driver.shaded.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) at com.datastax.oss.driver.shaded.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818) at com.datastax.oss.driver.shaded.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at com.datastax.oss.driver.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497) at com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at com.datastax.oss.driver.shaded.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at com.datastax.oss.driver.shaded.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer

hopugop commented 1 year ago

This is an Astra Classic deployment (using nodes) cc @tarzanek