search

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8.01k stars 1.82k forks source link

[Bug] [Cdc] checkpoint is always increment #5157

Closed WhatAKitty closed 1 year ago

WhatAKitty commented 1 year ago

Search before asking

What happened

wait checkpoint completed: 1 ..... wait checkpoint completed: 204 wait checkpoint completed: 205 ..... and so on.

The bellow is some detailed msg:

2023-07-26 13:48:12,148 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:12,148 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:15,960 INFO  com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask - [ds01]:5801 [seatunnel] [5.1] Received auth from Connection[id=2, /10.200.1.89:5801->/10.200.1.90:43203, qualifier=null, endpoint=[10.200.1.90]:43203, remoteUuid=93bb2f95-0e06-4af3-9a3a-3e3b797c2bbb, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 93bb2f95-0e06-4af3-9a3a-3e3b797c2bbb, client name: hz.client_1, client version: 5.1
2023-07-26 13:48:17,024 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Init JobMaster for Job SeaTunnel_Job (736459787658592257)
2023-07-26 13:48:17,024 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job SeaTunnel_Job (736459787658592257) needed jar urls [file:/seatunnel/connector-cdc-mysql-2.3.3-SNAPSHOT.jar]
2023-07-26 13:48:17,055 DEBUG org.apache.seatunnel.engine.server.dag.execution.ExecutionPlanGenerator - Generate execution plan using logical plan:
2023-07-26 13:48:17,055 DEBUG org.apache.seatunnel.engine.server.dag.execution.ExecutionPlanGenerator - Phase 1: generate execution edge list [ExecutionEdge(leftVertex=ExecutionVertex(vertexId=1, action=org.apache.seatunnel.engine.core.dag.actions.SourceAction@2a502f47, parallelism=1), rightVertex=ExecutionVertex(vertexId=2, action=org.apache.seatunnel.engine.core.dag.actions.SinkAction@54d7c7e8, parallelism=1), leftVertexId=1, rightVertexId=2)]
2023-07-26 13:48:17,055 DEBUG org.apache.seatunnel.engine.server.dag.execution.ExecutionPlanGenerator - Phase 2: generate shuffle edge list [ExecutionEdge(leftVertex=ExecutionVertex(vertexId=1, action=org.apache.seatunnel.engine.core.dag.actions.SourceAction@2a502f47, parallelism=1), rightVertex=ExecutionVertex(vertexId=2, action=org.apache.seatunnel.engine.core.dag.actions.SinkAction@54d7c7e8, parallelism=1), leftVertexId=1, rightVertexId=2)]
2023-07-26 13:48:17,055 DEBUG org.apache.seatunnel.engine.server.dag.execution.ExecutionPlanGenerator - Phase 3: generate transform chain edge list [ExecutionEdge(leftVertex=ExecutionVertex(vertexId=1, action=org.apache.seatunnel.engine.core.dag.actions.SourceAction@2a502f47, parallelism=1), rightVertex=ExecutionVertex(vertexId=2, action=org.apache.seatunnel.engine.core.dag.actions.SinkAction@54d7c7e8, parallelism=1), leftVertexId=1, rightVertexId=2)]
2023-07-26 13:48:17,055 DEBUG org.apache.seatunnel.engine.server.dag.execution.ExecutionPlanGenerator - Phase 4: generate pipeline list [org.apache.seatunnel.engine.server.dag.execution.Pipeline@35024291]
2023-07-26 13:48:17,055 DEBUG org.apache.seatunnel.engine.server.dag.execution.ExecutionPlanGenerator - Phase 5: generate execution plan: org.apache.seatunnel.engine.server.dag.execution.ExecutionPlan@a22fef3
2023-07-26 13:48:17,064 INFO  org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage - Path /tmp/seatunnel/checkpoint_snapshot/736459787658592257 is not a directory
2023-07-26 13:48:17,064 INFO  org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage - No checkpoint found for job, job id is: 736459787658592257
2023-07-26 13:48:17,064 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}] is in state CREATED when init state future
2023-07-26 13:48:17,064 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}] is in state CREATED when init state future
2023-07-26 13:48:17,065 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job Job SeaTunnel_Job (736459787658592257) waiting for scheduler finished
2023-07-26 13:48:17,065 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (736459787658592257) turn from state CREATED to SCHEDULED.
2023-07-26 13:48:17,066 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)] turn from state CREATED to SCHEDULED.
2023-07-26 13:48:17,066 DEBUG org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}] state from CREATED to SCHEDULED
2023-07-26 13:48:17,067 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}] turn from state CREATED to SCHEDULED.
2023-07-26 13:48:17,067 DEBUG org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}] state from CREATED to SCHEDULED
2023-07-26 13:48:17,067 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot request, jobID: 736459787658592257, resource profile: ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, return: SlotProfile{worker=[ds01]:5801, slotID=3, ownerJobID=736459787658592257, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='f39eac91-0fc0-4dc9-bcaf-64f8d55dee79'}
2023-07-26 13:48:17,067 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}] turn from state CREATED to SCHEDULED.
2023-07-26 13:48:17,068 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot request, jobID: 736459787658592257, resource profile: ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, return: SlotProfile{worker=[ds01]:5801, slotID=4, ownerJobID=736459787658592257, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='f39eac91-0fc0-4dc9-bcaf-64f8d55dee79'}
2023-07-26 13:48:17,068 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:17,068 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:17,068 DEBUG org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler - slotProfiles: {TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}=SlotProfile{worker=[ds01]:5801, slotID=3, ownerJobID=736459787658592257, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='f39eac91-0fc0-4dc9-bcaf-64f8d55dee79'}, TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}=SlotProfile{worker=[ds01]:5801, slotID=4, ownerJobID=736459787658592257, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='f39eac91-0fc0-4dc9-bcaf-64f8d55dee79'}}, PipelineLocation: PipelineLocation(jobId=736459787658592257, pipelineId=1)
2023-07-26 13:48:17,075 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)] turn from state SCHEDULED to DEPLOYING.
2023-07-26 13:48:17,076 DEBUG org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}] state from SCHEDULED to DEPLOYING
2023-07-26 13:48:17,077 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}] turn from state SCHEDULED to DEPLOYING.
2023-07-26 13:48:17,077 DEBUG org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}] state from SCHEDULED to DEPLOYING
2023-07-26 13:48:17,077 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}] turn from state SCHEDULED to DEPLOYING.
2023-07-26 13:48:17,078 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] received deploying task executionId [736459461644124161]
2023-07-26 13:48:17,079 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] received deploying task executionId [736459461644189697]
2023-07-26 13:48:17,116 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] deploying task TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}, executionId [736459461644124161]
2023-07-26 13:48:17,116 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] deploying TaskGroup TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} init success
2023-07-26 13:48:17,117 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - starting seatunnel source split enumerator task, source name: pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]
2023-07-26 13:48:17,118 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] deploying TaskGroup TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} success
2023-07-26 13:48:17,118 DEBUG org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}] state from DEPLOYING to RUNNING
2023-07-26 13:48:17,118 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}] turn from state DEPLOYING to RUNNING.
2023-07-26 13:48:17,124 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported task(20000) status WAITING_RESTORE
2023-07-26 13:48:17,125 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : NotifyTaskRestoreOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}
2023-07-26 13:48:17,125 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}
2023-07-26 13:48:17,125 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] accept async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} with id 45c7e733-54f0-49c9-bb58-fb79c07df71e
2023-07-26 13:48:17,125 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation.restoreState []
2023-07-26 13:48:17,125 DEBUG org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - restoreState for split enumerator [[]]
2023-07-26 13:48:17,128 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] deploying task TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, executionId [736459461644189697]
2023-07-26 13:48:17,128 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] deploying TaskGroup TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} init success
2023-07-26 13:48:17,135 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] deploying TaskGroup TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} success
2023-07-26 13:48:17,135 DEBUG org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}] state from DEPLOYING to RUNNING
2023-07-26 13:48:17,136 INFO  org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting seatunnel transform task, index 0
2023-07-26 13:48:17,143 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported task(50000) status WAITING_RESTORE
2023-07-26 13:48:17,143 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : NotifyTaskRestoreOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}
2023-07-26 13:48:17,144 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}
2023-07-26 13:48:17,144 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] accept async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} with id 991487d5-4230-43b7-99bd-3a2d7db36a4f
2023-07-26 13:48:17,144 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation.restoreState []
2023-07-26 13:48:17,144 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - restoreState for SeaTunnelTask[[]]
2023-07-26 13:48:17,145 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}] turn from state DEPLOYING to RUNNING.
2023-07-26 13:48:17,145 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (736459787658592257), Pipeline: [(1/1)] turn from state DEPLOYING to RUNNING.
2023-07-26 13:48:17,146 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (736459787658592257) turn from state SCHEDULED to RUNNING.
2023-07-26 13:48:17,148 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job SeaTunnel_Job (736459787658592257) scheduler finished
2023-07-26 13:48:17,151 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:17,151 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:17,162 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - restoreState for SeaTunnelTask finished, actionStateList: []
2023-07-26 13:48:17,162 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation.finished []
2023-07-26 13:48:17,162 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] remove async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} with id 991487d5-4230-43b7-99bd-3a2d7db36a4f
2023-07-26 13:48:17,189 INFO  org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting seatunnel source task, index 0
2023-07-26 13:48:17,190 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported task(40000) status WAITING_RESTORE
2023-07-26 13:48:17,190 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : NotifyTaskRestoreOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}
2023-07-26 13:48:17,190 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2023-07-26 13:48:17,190 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] accept async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} with id d8f7052c-2e02-4872-a164-52609b82df3a
2023-07-26 13:48:17,191 INFO  com.zaxxer.hikari.HikariDataSource - connection-pool-10.200.1.78:33306 - Starting...
2023-07-26 13:48:17,192 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation.restoreState []
2023-07-26 13:48:17,192 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - restoreState for SeaTunnelTask[[]]
2023-07-26 13:48:17,192 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - restoreState for SeaTunnelTask finished, actionStateList: []
2023-07-26 13:48:17,192 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation.finished []
2023-07-26 13:48:17,192 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] remove async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} with id d8f7052c-2e02-4872-a164-52609b82df3a
2023-07-26 13:48:17,244 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported task(50000) status READY_START
2023-07-26 13:48:17,291 INFO  org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase - Open Source Reader.
2023-07-26 13:48:17,291 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] accept async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} with id 2fc2402a-ecbe-4b5b-a850-1e1ca2620b99
2023-07-26 13:48:17,291 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2023-07-26 13:48:17,294 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported task(40000) status READY_START
2023-07-26 13:48:17,319 INFO  com.zaxxer.hikari.HikariDataSource - connection-pool-10.200.1.78:33306 - Start completed.
2023-07-26 13:48:17,343 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.utils.TableDiscoveryUtils - Read list of available databases
2023-07-26 13:48:17,364 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.utils.TableDiscoveryUtils -    list of available databases is: [information_schema, cdc_test]
2023-07-26 13:48:17,364 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.utils.TableDiscoveryUtils - Read list of available tables in each database
2023-07-26 13:48:17,378 INFO  io.debezium.jdbc.JdbcConnection - Connection gracefully closed
2023-07-26 13:48:17,389 INFO  io.debezium.jdbc.JdbcConnection - Connection gracefully closed
2023-07-26 13:48:17,392 DEBUG org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - restoreState split enumerator [[]] finished
2023-07-26 13:48:17,392 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation - NotifyTaskRestoreOperation.finished []
2023-07-26 13:48:17,392 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] remove async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} with id 45c7e733-54f0-49c9-bb58-fb79c07df71e
2023-07-26 13:48:17,398 DEBUG org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - reader register complete, current task size 1
2023-07-26 13:48:17,398 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] remove async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} with id 2fc2402a-ecbe-4b5b-a850-1e1ca2620b99
2023-07-26 13:48:17,425 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported task(20000) status READY_START
2023-07-26 13:48:17,425 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : NotifyTaskStartOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}
2023-07-26 13:48:17,426 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : NotifyTaskStartOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}
2023-07-26 13:48:17,426 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : NotifyTaskStartOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}
2023-07-26 13:48:17,526 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received enough reader, starting enumerator...
2023-07-26 13:48:17,527 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.eumerator.MySqlChunkSplitter - Start splitting table cdc_test.test_1 into chunks...
2023-07-26 13:48:17,546 INFO  io.debezium.jdbc.JdbcConnection - Connection gracefully closed
2023-07-26 13:48:17,566 WARN  io.debezium.relational.history.DatabaseHistoryMetrics - Unable to register the MBean 'debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source': debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source
2023-07-26 13:48:18,122 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.eumerator.MySqlChunkSplitter - The distribution factor of table cdc_test.test_1 is 1.0 according to the min split key 1, max split key 2 and approximate row count 2
2023-07-26 13:48:18,122 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.eumerator.MySqlChunkSplitter - Use evenly-sized chunk optimization for table cdc_test.test_1, the approximate row count is 2, the chunk size is 8096, the dynamic chunk size is 8096
2023-07-26 13:48:18,123 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.eumerator.MySqlChunkSplitter - Split table cdc_test.test_1 into 1 chunks, time cost: 595ms.
2023-07-26 13:48:18,123 INFO  io.debezium.jdbc.JdbcConnection - Connection gracefully closed
2023-07-26 13:48:18,132 INFO  org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher - Starting split fetcher 0
2023-07-26 13:48:18,162 WARN  io.debezium.connector.mysql.MySqlConnection - Database configuration option 'serverTimezone' is set but is obsolete, please use 'connectionTimeZone' instead
2023-07-26 13:48:18,186 INFO  com.zaxxer.hikari.HikariDataSource - connection-pool-10.200.1.78:33306 - Starting...
2023-07-26 13:48:18,192 WARN  com.zaxxer.hikari.util.DriverDataSource - Registered driver with driverClassName=com.mysql.cj.jdbc.Driver was not found, trying direct instantiation.
2023-07-26 13:48:18,214 INFO  com.zaxxer.hikari.HikariDataSource - connection-pool-10.200.1.78:33306 - Start completed.
2023-07-26 13:48:18,228 INFO  io.debezium.jdbc.JdbcConnection - Connection gracefully closed
2023-07-26 13:48:18,252 WARN  io.debezium.relational.history.DatabaseHistoryMetrics - Unable to register the MBean 'debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source': debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source
2023-07-26 13:48:18,949 WARN  io.debezium.relational.history.DatabaseHistoryMetrics - Unable to register the MBean 'debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source': debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source
2023-07-26 13:48:18,950 INFO  io.debezium.relational.history.DatabaseHistoryMetrics - Started database history recovery
2023-07-26 13:48:18,950 INFO  io.debezium.relational.history.DatabaseHistoryMetrics - Finished database history recovery of 0 change(s) in 0 ms
2023-07-26 13:48:19,461 INFO  org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher - Finished reading from splits [cdc_test.test_1:0]
2023-07-26 13:48:19,499 INFO  org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase - Finished reading split(s) [cdc_test.test_1:0]
2023-07-26 13:48:19,503 INFO  org.apache.seatunnel.connectors.cdc.base.source.enumerator.SnapshotSplitAssigner - Snapshot split assigner received all splits completed and the job parallelism is 1, snapshot split assigner is turn into completed status.
2023-07-26 13:48:19,509 INFO  org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceSplitReader - It's turn to read incremental split, close current snapshot fetcher.
2023-07-26 13:48:19,511 INFO  io.debezium.jdbc.JdbcConnection - Connection gracefully closed
2023-07-26 13:48:19,512 WARN  io.debezium.connector.mysql.MySqlConnection - Database configuration option 'serverTimezone' is set but is obsolete, please use 'connectionTimeZone' instead
2023-07-26 13:48:19,513 INFO  org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceSplitReader - Stream fetcher is created.
2023-07-26 13:48:19,530 WARN  io.debezium.relational.history.DatabaseHistoryMetrics - Unable to register the MBean 'debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source': debezium.mysql:type=connector-metrics,context=schema-history,server=mysql_binlog_source
2023-07-26 13:48:19,531 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.reader.fetch.MySqlSourceFetchTaskContext - MySQL has the binlog file 'mysql-bin.000001' required by the connector
2023-07-26 13:48:19,531 INFO  io.debezium.relational.history.DatabaseHistoryMetrics - Started database history recovery
2023-07-26 13:48:19,531 INFO  io.debezium.relational.history.DatabaseHistoryMetrics - Finished database history recovery of 0 change(s) in 0 ms
2023-07-26 13:48:19,538 INFO  io.debezium.util.Threads - Requested thread factory for connector MySqlConnector, id = mysql_binlog_source named = binlog-client
2023-07-26 13:48:19,551 INFO  org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.FieldNamedPreparedStatement - PrepareStatement sql is:
INSERT INTO `cdc_test`.`test_1_target` (`id`, `name`) VALUES (?, ?)

2023-07-26 13:48:19,558 INFO  io.debezium.util.Threads - Creating thread debezium-mysqlconnector-mysql_binlog_source-binlog-client
2023-07-26 13:48:19,561 INFO  io.debezium.util.Threads - Creating thread debezium-mysqlconnector-mysql_binlog_source-binlog-client
2023-07-26 13:48:19,569 INFO  io.debezium.util.Threads - Creating thread debezium-mysqlconnector-mysql_binlog_source-binlog-client
2023-07-26 13:48:22,142 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1]
***********************************************
    TaskExecutionServer Thread Pool Status
***********************************************
activeCount               :                   4
threadShareTaskQueueSize  :                   0
completedTaskCount        :                  37
taskCount                 :                  41
***********************************************

2023-07-26 13:48:22,148 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:22,148 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:22,151 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [ds01]:5801 [seatunnel] [5.1]
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   1
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   6
completedTaskCount        :                  68
taskCount                 :                  69
***********************************************

2023-07-26 13:48:22,152 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [ds01]:5801 [seatunnel] [5.1]
***********************************************
                Job info detail
***********************************************
createdJobCount           :                   0
scheduledJobCount         :                   0
runningJobCount           :                   1
failingJobCount           :                   0
failedJobCount            :                   0
cancellingJobCount        :                   0
canceledJobCount          :                   0
finishedJobCount          :                   0
restartingJobCount        :                   0
suspendedJobCount         :                   0
reconcilingJobCount       :                   0
***********************************************

2023-07-26 13:48:27,148 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:27,148 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:32,141 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1]
***********************************************
    TaskExecutionServer Thread Pool Status
***********************************************
activeCount               :                   4
threadShareTaskQueueSize  :                   0
completedTaskCount        :                  37
taskCount                 :                  41
***********************************************

2023-07-26 13:48:32,148 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:32,148 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:32,427 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - wait checkpoint completed: 1
2023-07-26 13:48:32,427 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - trigger checkpoint barrier 736459787658592257/1/1, CHECKPOINT_TYPE
2023-07-26 13:48:32,428 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : CheckpointBarrierTriggerOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}
2023-07-26 13:48:32,428 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Start a scheduled task to prevent checkpoint timeouts for barrier 736459787658592257/1/1, CHECKPOINT_TYPE
2023-07-26 13:48:32,428 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] accept async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} with id 45e2811f-81a1-4984-9657-0ae58264b6d1
2023-07-26 13:48:32,428 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointBarrierTriggerOperation - CheckpointBarrierTriggerOperation [TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}]
2023-07-26 13:48:32,428 DEBUG org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - split enumer trigger barrier [CheckpointBarrier 1 @ 1690350512427 Options: CHECKPOINT_TYPE]
2023-07-26 13:48:32,430 DEBUG org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - source split enumerator send state [HybridPendingSplitsState(snapshotPhaseState=SnapshotPhaseState(remainingTables=[], alreadyProcessedTables=[cdc_test.test_1], remainingSplits=[], assignedSplits={cdc_test.test_1:0=org.apache.seatunnel.connectors.cdc.base.source.split.SnapshotSplit@f041ba98}, splitCompletedOffsets={cdc_test.test_1:0={ts_sec=0, file=mysql-bin.000001, pos=12566283, gtids=209753da-221a-11ee-9983-76b78ab3e808:1-51705, row=0, event=0}}, isAssignerCompleted=true, isTableIdCaseSensitive=false, isRemainingTablesCheckpointed=true), incrementalPhaseState=IncrementalPhaseState())] to master
2023-07-26 13:48:32,430 DEBUG org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - split enumerator send to read--size: 1, location: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}, address: [ds01]:5801
2023-07-26 13:48:32,430 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] accept async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} with id 8ddaf358-db77-401f-a336-992f4398ea7c
2023-07-26 13:48:32,430 DEBUG org.apache.seatunnel.engine.server.task.operation.checkpoint.BarrierFlowOperation - CheckpointBarrierTriggerOperation [TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}]
2023-07-26 13:48:32,430 DEBUG org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle - source trigger barrier [CheckpointBarrier 1 @ 1690350512427 Options: CHECKPOINT_TYPE]
2023-07-26 13:48:32,430 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.TaskAcknowledgeOperation - TaskAcknowledgeOperation TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - checkpoint manager received ack TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - task[20000](1/736459787658592257) ack. CheckpointBarrier 1 @ 1690350512427 Options: CHECKPOINT_TYPE
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint - acknowledgeTask states [[ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]), index=-1, state=[[B@dc476b7])]]
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.TaskAcknowledgeOperation - task ack finished TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}, taskID=20000, index=0}
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] remove async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1} with id 45e2811f-81a1-4984-9657-0ae58264b6d1
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - seatunnel task ack barrier[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}]
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle - source ack barrier finished, taskId: [40000]
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - seatunnel task ack barrier[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}]
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.TaskAcknowledgeOperation - TaskAcknowledgeOperation TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - checkpoint manager received ack TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - task[40000](1/736459787658592257) ack. CheckpointBarrier 1 @ 1690350512427 Options: CHECKPOINT_TYPE
2023-07-26 13:48:32,431 DEBUG org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint - acknowledgeTask states [[ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]), index=0, state=[[B@79f1a229])]]
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.TaskAcknowledgeOperation - task ack finished TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle - send record to next finished, taskId: [40000]
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - seatunnel task ack barrier[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}]
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1] remove async execute function from TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000} with id 8ddaf358-db77-401f-a336-992f4398ea7c
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.task.SeaTunnelTask - seatunnel task ack barrier[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}]
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.TaskAcknowledgeOperation - TaskAcknowledgeOperation TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - checkpoint manager received ack TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - task[50000](1/736459787658592257) ack. CheckpointBarrier 1 @ 1690350512427 Options: CHECKPOINT_TYPE
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint - acknowledgeTask states [[ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-jdbc-cdc_test.test_1]), index=0, state=[])]]
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint - checkpoint is full ack!
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.operation.TaskAcknowledgeOperation - task ack finished TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}, taskID=50000, index=0}
2023-07-26 13:48:32,432 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - pending checkpoint(1/1@736459787658592257) completed! cost: 5, trigger: 1690350512427, completed: 1690350512432
2023-07-26 13:48:32,473 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - pending checkpoint(1/1@736459787658592257) notify finished!
2023-07-26 13:48:32,474 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start notify checkpoint completed, checkpoint:CompletedCheckpoint(jobId=736459787658592257, pipelineId=1, checkpointId=1, triggerTimestamp=1690350512427, checkpointType=CHECKPOINT_TYPE, completedTimestamp=1690350512432, taskStates={ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-jdbc-cdc_test.test_1])=ActionState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-jdbc-cdc_test.test_1]), subtaskStates=[ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-jdbc-cdc_test.test_1]), index=0, state=[])], coordinatorState=null, parallelism=1), ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-MySQL-CDC-test_1_shared])=ActionState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]), subtaskStates=[ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]), index=0, state=[[B@79f1a229])], coordinatorState=ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-MySQL-CDC-test_1_shared]), index=-1, state=[[B@dc476b7]), parallelism=1)}, taskStatistics={2=TaskStatistics(jobVertexId=2, subtaskStats=[SubtaskStatistics(subtaskIndex=0, ackTimestamp=1690350512431, stateSize=1, subtaskStatus=RUNNING)], subtaskCompleted=[false], numAcknowledgedSubtasks=1, latestAckedSubtaskStatistics=null), 4=TaskStatistics(jobVertexId=4, subtaskStats=[SubtaskStatistics(subtaskIndex=0, ackTimestamp=1690350512432, stateSize=1, subtaskStatus=RUNNING)], subtaskCompleted=[false], numAcknowledgedSubtasks=1, latestAckedSubtaskStatistics=null), 5=TaskStatistics(jobVertexId=5, subtaskStats=[SubtaskStatistics(subtaskIndex=0, ackTimestamp=1690350512432, stateSize=0, subtaskStatus=RUNNING)], subtaskCompleted=[false], numAcknowledgedSubtasks=1, latestAckedSubtaskStatistics=null)})
2023-07-26 13:48:32,475 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : CheckpointFinishedOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}
2023-07-26 13:48:32,477 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : CheckpointFinishedOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=30000}
2023-07-26 13:48:32,477 DEBUG org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - Sead Operation : CheckpointFinishedOperation to [ds01]:5801 for task group:TaskGroupLocation{jobId=736459787658592257, pipelineId=1, taskGroupId=1}
2023-07-26 13:48:37,148 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:37,148 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:42,140 DEBUG org.apache.seatunnel.engine.server.TaskExecutionService - [ds01]:5801 [seatunnel] [5.1]
***********************************************
    TaskExecutionServer Thread Pool Status
***********************************************
activeCount               :                   4
threadShareTaskQueueSize  :                   0
completedTaskCount        :                  39
taskCount                 :                  43
***********************************************

2023-07-26 13:48:42,148 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:42,148 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:47,148 DEBUG org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - start send heartbeat to resource manager, this address: [ds01]:5801
2023-07-26 13:48:47,148 DEBUG org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - received worker heartbeat from: [ds01]:5801
2023-07-26 13:48:47,428 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - wait checkpoint completed: 2

SeaTunnel Version

2.3.2

SeaTunnel Config

seatunnel:
  engine:
    backup-count: 1
    queue-type: blockingqueue
    print-execution-info-interval: 60
    print-job-metrics-info-interval: 60
    slot-service:
      dynamic-slot: true
    checkpoint:
      interval: 10000
      timeout: 60000
      max-concurrent: 1
      tolerable-failure: 2
      storage:
        type: hdfs
        max-retained: 3
        plugin-config:
          namespace: /tmp/seatunnel/checkpoint_snapshot
          storage.type: hdfs
          fs.defaultFS: file:///tmp

Running Command

./bin/seatunnel-cluster.sh -d

Error Exception

Nothing

Flink or Spark Version

No response

Java or Scala Version

1.8.0_191

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

wu-a-ge commented 1 year ago

it's normal,but you should apply a patch #5054