search

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.97k stars 1.8k forks source link

STARROCKS-06 #7473

Closed fzzr1101 closed 3 weeks ago

fzzr1101 commented 2 months ago

Search before asking

What happened

I want to read data from starrocks,before that i can read data from mysql and write data into starrocks,only reading starrocks exists problem.

SeaTunnel Version

2.3.6

SeaTunnel Config

env {
  parallelism = 4
  job.mode = "BATCH"
}
source {
    StarRocks {
        nodeUrls = ["172.26.48.1:9030"]
        username = "myuser"
        password = "mypassword"
        database = "mydb"
        table = "mytable"
        schema  {
            fields  {
             name = STRING
             age = INT
            }
        }
    }
}
sink {
    console{
    }
}

Running Command

2024-08-23 15:24:27,719 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] Copyright © 2021-2022 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2024-08-23 15:24:27,720 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following:
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2024-08-23 15:24:27,722 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2024-08-23 15:24:27,966 INFO  [c.h.s.security                ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see 🔒 security recommendations and the status of current config.
2024-08-23 15:24:28,010 INFO  [o.a.s.e.s.SeaTunnelNodeContext] [main] - Using LiteNodeDropOutTcpIpJoiner TCP/IP discovery
2024-08-23 15:24:28,012 WARN  [c.h.c.CPSubsystem             ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2024-08-23 15:24:28,128 INFO  [.c.c.DefaultClassLoaderService] [main] - start classloader service
2024-08-23 15:24:28,131 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/home/xcj/apache-seatunnel-2.3.6/config/seatunnel.yaml' from System property 'seatunnel.config'
2024-08-23 15:24:28,131 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /home/xcj/apache-seatunnel-2.3.6/config/seatunnel.yaml
2024-08-23 15:24:28,134 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/home/xcj/apache-seatunnel-2.3.6/config/hazelcast.yaml' from System property 'hazelcast.config'
2024-08-23 15:24:28,134 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /home/xcj/apache-seatunnel-2.3.6/config/hazelcast.yaml
2024-08-23 15:24:28,140 WARN  [o.a.s.e.s.TaskExecutionService] [pool-2-thread-1] - [localhost]:5801 [seatunnel-328872] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2024-08-23 15:24:28,141 INFO  [o.a.s.e.s.TaskExecutionService] [main] - [localhost]:5801 [seatunnel-328872] [5.1] Created new BusWork : 658563862
2024-08-23 15:24:28,156 WARN  [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2024-08-23 15:24:28,159 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-6-thread-1] - [localhost]:5801 [seatunnel-328872] [5.1]
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2024-08-23 15:24:28,183 INFO  [c.h.i.d.Diagnostics           ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2024-08-23 15:24:28,186 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5801 is STARTING
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.hazelcast.internal.networking.nio.SelectorOptimizer (file:/home/xcj/apache-seatunnel-2.3.6/starter/seatunnel-starter.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of com.hazelcast.internal.networking.nio.SelectorOptimizer
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-2] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5821 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-11] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5810 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-3] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5823 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5827 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-6] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5813 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-9] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5806 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-5] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5815 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5804 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5824 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-6] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5812 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-5] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5818 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-14] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5829 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5807 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5805 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-6] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5811 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-14] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5809 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-16] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5820 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-13] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5802 is added to the blacklist.
2024-08-23 15:24:28,217 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-8] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5817 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5828 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5830 is added to the blacklist.
2024-08-23 15:24:28,219 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-6] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5803 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-11] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5816 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-2] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5826 is added to the blacklist.
2024-08-23 15:24:28,219 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-12] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5808 is added to the blacklist.
2024-08-23 15:24:28,219 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-15] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5822 is added to the blacklist.
2024-08-23 15:24:28,219 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-7] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5819 is added to the blacklist.
2024-08-23 15:24:28,218 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-3] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5814 is added to the blacklist.
2024-08-23 15:24:28,219 INFO  [e.s.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-4] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5825 is added to the blacklist.
2024-08-23 15:24:29,211 INFO  [c.h.i.c.ClusterService        ] [main] - [localhost]:5801 [seatunnel-328872] [5.1]

Members {size:1, ver:1} [
        Member [localhost]:5801 - 7544e8a5-7b1c-4b60-8a4b-14998f789010 this master
]

2024-08-23 15:24:29,218 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-328872] [5.1] [localhost]:5801 is STARTED
2024-08-23 15:24:29,241 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel-328872] [5.1] Running with 2 response threads, dynamic=true
2024-08-23 15:24:29,247 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-328872] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2024-08-23 15:24:29,248 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-328872] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2024-08-23 15:24:29,251 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-328872] [5.1] Trying to connect to cluster: seatunnel-328872
2024-08-23 15:24:29,252 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-328872] [5.1] Trying to connect to [localhost]:5801
2024-08-23 15:24:29,257 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-328872] [5.1] This node become a new active master node, begin init coordinator service
2024-08-23 15:24:29,262 INFO  [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [localhost]:5801 [seatunnel-328872] [5.1] Received auth from Connection[id=1, /127.0.0.1:5801->/127.0.0.1:37799, qualifier=null, endpoint=[127.0.0.1]:37799, remoteUuid=fd3cb69c-cf76-423e-9346-c5ef8553c567, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: fd3cb69c-cf76-423e-9346-c5ef8553c567, client name: hz.client_1, client version: 5.1
2024-08-23 15:24:29,263 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-328872] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2024-08-23 15:24:29,263 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-328872] [5.1] Authenticated with server [localhost]:5801:7544e8a5-7b1c-4b60-8a4b-14998f789010, server version: 5.1, local address: /127.0.0.1:37799
2024-08-23 15:24:29,264 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel-328872] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2024-08-23 15:24:29,268 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-7] - hz.client_1 [seatunnel-328872] [5.1]

Members [1] {
        Member [localhost]:5801 - 7544e8a5-7b1c-4b60-8a4b-14998f789010
}

2024-08-23 15:24:29,285 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2024-08-23 15:24:29,368 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-328872] [5.1] Loaded event handlers: [org.apache.seatunnel.api.event.LoggingEventHandler@59df5422]
2024-08-23 15:24:29,390 INFO  [.h.i.p.i.PartitionStateManager] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] Initializing cluster partition table arrangement...
2024-08-23 15:24:29,398 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Loading config file from path: test.conf
2024-08-23 15:24:29,429 INFO  [o.a.s.c.s.u.ConfigShadeUtils  ] [main] - Load config shade spi: [base64]
2024-08-23 15:24:29,468 INFO  [p.MultipleTableJobConfigParser] [main] - add common jar in plugins :[]
2024-08-23 15:24:29,477 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /home/xcj/apache-seatunnel-2.3.6/connectors
2024-08-23 15:24:29,479 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='StarRocks'} at: file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar
2024-08-23 15:24:29,479 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Console'} at: file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-console-2.3.6.jar
2024-08-23 15:24:29,481 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2024-08-23 15:24:29,517 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - get the CatalogTable from source StarRocks: .default.default.default
2024-08-23 15:24:29,526 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from /home/xcj/apache-seatunnel-2.3.6/connectors
2024-08-23 15:24:29,532 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='StarRocks'} at: file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar
2024-08-23 15:24:29,534 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all transforms.
2024-08-23 15:24:29,534 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sinks.
2024-08-23 15:24:29,537 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /home/xcj/apache-seatunnel-2.3.6/connectors
2024-08-23 15:24:29,538 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Console'} at: file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-console-2.3.6.jar
2024-08-23 15:24:29,541 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Console' with upstream input catalog-table[database: default, schema: default, table: default]
2024-08-23 15:24:29,560 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Start submit job, job id: 879264821487337473, with plugin jar [file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-console-2.3.6.jar, file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar]
2024-08-23 15:24:29,565 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - Init ResourceManager
2024-08-23 15:24:29,565 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - initWorker...
2024-08-23 15:24:29,565 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - init live nodes: [[localhost]:5801]
2024-08-23 15:24:29,566 INFO  [.e.s.r.AbstractResourceManager] [ForkJoinPool.commonPool-worker-19] - received new worker register: [localhost]:5801
2024-08-23 15:24:29,566 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=536870912}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=536870912}}, dynamicSlot=true, assignedSlots=[], unassignedSlots=[], attributes={})}
2024-08-23 15:24:29,575 INFO  [o.a.s.e.s.m.JobMaster         ] [seatunnel-coordinator-service-0] - in batch mode, the 'checkpoint.interval' configuration of env is missing, so checkpoint will be disabled
2024-08-23 15:24:29,575 INFO  [o.a.s.e.s.m.JobMaster         ] [seatunnel-coordinator-service-0] - Init JobMaster for Job SeaTunnel_Job (879264821487337473)
2024-08-23 15:24:29,575 INFO  [o.a.s.e.s.m.JobMaster         ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473) needed jar urls [file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-console-2.3.6.jar, file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar]
2024-08-23 15:24:29,575 INFO  [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-0] - Create classloader for job 879264821487337473 with jars [file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-console-2.3.6.jar, file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar]
2024-08-23 15:24:29,610 INFO  [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-0] - Release classloader for job 879264821487337473 with jars [file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-console-2.3.6.jar, file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar]
2024-08-23 15:24:29,770 WARN  [o.a.h.u.NativeCodeLoader      ] [seatunnel-coordinator-service-0] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2024-08-23 15:24:29,890 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-0] - Create CheckpointCoordinator for job(1@879264821487337473) with plan(CheckpointPlan(pipelineId=1, pipelineSubtasks=[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}, TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}, TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}], startingSubtasks=[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}], pipelineActions={ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-StarRocks])=1, ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-Console-MultiTableSink])=1}, subtaskActions={TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-StarRocks]), 0)], TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-Console-MultiTableSink]), 0)], TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-StarRocks]), -1)]}))
2024-08-23 15:24:29,891 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - The task Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SourceTask (1/1)] is in state CREATED when init state future
2024-08-23 15:24:29,892 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - The task Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SplitEnumerator (1/1)] is in state CREATED when init state future
2024-08-23 15:24:29,893 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473) state process is start
2024-08-23 15:24:29,894 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Submit job finished, job id: 879264821487337473, job name: SeaTunnel_Job
2024-08-23 15:24:29,897 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473) turned from state CREATED to SCHEDULED.
2024-08-23 15:24:29,897 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)] state process is start
2024-08-23 15:24:29,899 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)] turned from state CREATED to SCHEDULED.
2024-08-23 15:24:29,906 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)] turned from state SCHEDULED to DEPLOYING.
2024-08-23 15:24:29,906 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SplitEnumerator (1/1)] state process is start
2024-08-23 15:24:29,907 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SplitEnumerator (1/1)] turned from state CREATED to DEPLOYING.
2024-08-23 15:24:29,912 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] received deploying task executionId [879264823647404033]
2024-08-23 15:24:29,914 INFO  [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-0] - Create classloader for job 879264821487337473 with jars [file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar]
2024-08-23 15:24:29,918 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] deploying task TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}, executionId [879264823647404033]
2024-08-23 15:24:29,918 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] deploying TaskGroup TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1} init success
2024-08-23 15:24:29,921 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] deploying TaskGroup TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1} success
2024-08-23 15:24:29,921 WARN  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-879264821487337473] - Failed to get job metrics summary, it maybe first-run
2024-08-23 15:24:29,922 INFO  [.s.t.SourceSplitEnumeratorTask] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}] - starting seatunnel source split enumerator task, source name: pipeline-1 [Source[0]-StarRocks]
2024-08-23 15:24:29,923 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SplitEnumerator (1/1)] turned from state DEPLOYING to RUNNING.
2024-08-23 15:24:29,923 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SplitEnumerator (1/1)] current state equals target state: RUNNING, skip
2024-08-23 15:24:29,923 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SourceTask (1/1)] state process is start
2024-08-23 15:24:29,925 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SourceTask (1/1)] turned from state CREATED to DEPLOYING.
2024-08-23 15:24:29,933 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] received deploying task executionId [879264823647469569]
2024-08-23 15:24:29,933 INFO  [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-0] - Create classloader for job 879264821487337473 with jars [file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-console-2.3.6.jar, file:/home/xcj/apache-seatunnel-2.3.6/connectors/connector-starrocks-2.3.6.jar]
2024-08-23 15:24:29,939 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] deploying task TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}, executionId [879264823647469569]
2024-08-23 15:24:29,940 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] deploying TaskGroup TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000} init success
2024-08-23 15:24:29,941 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-0] - [localhost]:5801 [seatunnel-328872] [5.1] deploying TaskGroup TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000} success
2024-08-23 15:24:29,942 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SourceTask (1/1)] turned from state DEPLOYING to RUNNING.
2024-08-23 15:24:29,942 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-StarRocks]-SourceTask (1/1)] current state equals target state: RUNNING, skip
2024-08-23 15:24:29,943 INFO  [s.e.s.t.TransformSeaTunnelTask] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}] - starting seatunnel transform task, index 0
2024-08-23 15:24:29,943 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473), Pipeline: [(1/1)] turned from state DEPLOYING to RUNNING.
2024-08-23 15:24:29,946 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-0] - Job SeaTunnel_Job (879264821487337473) turned from state SCHEDULED to RUNNING.
2024-08-23 15:24:29,947 INFO  [.a.s.e.s.t.SourceSeaTunnelTask] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}] - starting seatunnel source task, index 0
2024-08-23 15:24:29,952 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [hz.main.seaTunnel.task.thread-4] - output rowType:
2024-08-23 15:24:29,954 INFO  [a.s.a.s.m.MultiTableSinkWriter] [hz.main.seaTunnel.task.thread-4] - init multi table sink writer, queue size: 1
2024-08-23 15:24:30,052 INFO  [.s.t.SourceSplitEnumeratorTask] [hz.main.seaTunnel.task.thread-4] - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2024-08-23 15:24:30,055 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-1] - checkpoint is disabled, because in batch mode and 'checkpoint.interval' of env is missing.
2024-08-23 15:24:30,125 INFO  [.s.t.SourceSplitEnumeratorTask] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}] - received enough reader, starting enumerator...
2024-08-23 15:24:30,136 INFO  [o.a.s.c.s.s.c.HttpHelper      ] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}] - Executing POST from http://172.26.48.1:9030/api/test/from_mysql/_query_plan.
2024-08-23 15:24:31,319 INFO  [o.a.s.c.s.s.c.HttpHelper      ] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}] - Executing POST from http://172.26.48.1:9030/api/test/from_mysql/_query_plan.
2024-08-23 15:24:32,335 INFO  [o.a.s.c.s.s.c.HttpHelper      ] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}] - Executing POST from http://172.26.48.1:9030/api/test/from_mysql/_query_plan.
2024-08-23 15:24:33,348 ERROR [s.StarRocksQueryPlanReadClient] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}] - Request query Plan From 172.26.48.1:9030 failed: Execute given execution failed after retry 3 times
2024-08-23 15:24:33,353 WARN  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=879264821487337473, pipelineId=1, taskGroupId=1}] - [localhost]:5801 [seatunnel-328872] [5.1] Exception in org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask@5d63b91a
org.apache.seatunnel.connectors.seatunnel.starrocks.exception.StarRocksConnectorException: ErrorCode:[STARROCKS-06], ErrorDescription:[Request query Plan failed] - query failed with empty response
        at org.apache.seatunnel.connectors.seatunnel.starrocks.client.source.StarRocksQueryPlanReadClient.getQueryPlan(StarRocksQueryPlanReadClient.java:166) ~[connector-starrocks-2.3.6.jar:2.3.6]
        at org.apache.seatunnel.connectors.seatunnel.starrocks.client.source.StarRocksQueryPlanReadClient.findPartitions(StarRocksQueryPlanReadClient.java:66) ~[connector-starrocks-2.3.6.jar:2.3.6]
        at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StartRocksSourceSplitEnumerator.getStarRocksSourceSplit(StartRocksSourceSplitEnumerator.java:184) ~[connector-starrocks-2.3.6.jar:2.3.6]
        at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StartRocksSourceSplitEnumerator.run(StartRocksSourceSplitEnumerator.java:79) ~[connector-starrocks-2.3.6.jar:2.3.6]
        at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.stateProcess(SourceSplitEnumeratorTask.java:319) ~[seatunnel-starter.jar:2.3.6]
        at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.call(SourceSplitEnumeratorTask.java:138) ~[seatunnel-starter.jar:2.3.6]
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:703) [seatunnel-starter.jar:2.3.6]
        at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1021) [seatunnel-starter.jar:2.3.6]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]

Error Exception

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:211)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.connectors.seatunnel.starrocks.exception.StarRocksConnectorException: ErrorCode:[STARROCKS-06], ErrorDescription:[Request query Plan failed] - query failed with empty response
        at org.apache.seatunnel.connectors.seatunnel.starrocks.client.source.StarRocksQueryPlanReadClient.getQueryPlan(StarRocksQueryPlanReadClient.java:166)
        at org.apache.seatunnel.connectors.seatunnel.starrocks.client.source.StarRocksQueryPlanReadClient.findPartitions(StarRocksQueryPlanReadClient.java:66)
        at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StartRocksSourceSplitEnumerator.getStarRocksSourceSplit(StartRocksSourceSplitEnumerator.java:184)
        at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StartRocksSourceSplitEnumerator.run(StartRocksSourceSplitEnumerator.java:79)
        at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.stateProcess(SourceSplitEnumeratorTask.java:319)
        at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.call(SourceSplitEnumeratorTask.java:138)
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:703)
        at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1021)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:203)
        ... 2 more
2024-08-23 15:24:52,468 INFO  [s.c.s.s.c.ClientExecuteCommand] [ForkJoinPool.commonPool-worker-5] - run shutdown hook because get close signal

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

zhoujiayingvana commented 2 months ago

设置端口为8030后，（nodeUrls = ["172.26.48.1:8030"]）报错为

 ErrorCode:[STARROCKS-04], ErrorDescription:[Create StarRocks BE reader failed] - Failed to open socket

具体为：

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:211)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.connectors.seatunnel.starrocks.exception.StarRocksConnectorException: ErrorCode:[STARROCKS-04], ErrorDescription:[Create StarRocks BE reader failed] - Failed to open socket
        at org.apache.seatunnel.connectors.seatunnel.starrocks.client.source.StarRocksBeReadClient.<init>(StarRocksBeReadClient.java:84)
        at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StarRocksSourceReader.read(StarRocksSourceReader.java:102)
        at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StarRocksSourceReader.pollNext(StarRocksSourceReader.java:65)
        at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:156)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:127)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:132)
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:703)
        at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1021)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:826)
Caused by: com.starrocks.shade.org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 3 weeks ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.