search

apache / flink-cdc

Flink CDC is a streaming data integration tool
https://nightlies.apache.org/flink/flink-cdc-docs-stable
Apache License 2.0
5.74k stars 1.95k forks source link

Flink CDC3.0 Demo Mysql binlog to Starrocks Exception amount of AddColumnEvent is already existed[Bug] #2910

Closed biyinggun closed 7 months ago

biyinggun commented 11 months ago

Search before asking

Flink version

1.18

Flink CDC version

3.0

Database and its version

Starrocks 3.1.6

Minimal reproduce step

I Can run the Doris Demo (mysq to doris)，But run Starrocks Demo Exception; For the Step: 1.Mysql Sync to Starrocks database and table pass.

  1. INSERT INTO app_db.orders (id, price) VALUES (3, 100.00); pass.
  2. ALTER TABLE app_db.orders ADD amount varchar(100) NULL; Flink CDC Exception :2023-12-21 11:40:16 java.lang.IllegalStateException: Failed to send request to coordinator: com.ververica.cdc.runtime.operators.schema.event.SchemaChangeRequest@ecbe495a .......Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: amount of AddColumnEvent is already existed. But Starrocks table: order.amount is added;

What did you expect to see?

2023-12-21 11:40:16 java.lang.IllegalStateException: Failed to send request to coordinator: com.ververica.cdc.runtime.operators.schema.event.SchemaChangeRequest@ecbe495a at com.ververica.cdc.runtime.operators.schema.SchemaOperator.sendRequestToCoordinator(SchemaOperator.java:126) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.requestSchemaChange(SchemaOperator.java:110) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.handleSchemaChangeEvent(SchemaOperator.java:95) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.processElement(SchemaOperator.java:85) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:309) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter$OutputCollector.collect(MySqlRecordEmitter.java:150) at java.util.ArrayList.forEach(ArrayList.java:1259) at com.ververica.cdc.debezium.event.DebeziumEventDeserializationSchema.deserialize(DebeziumEventDeserializationSchema.java:92) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitElement(MySqlRecordEmitter.java:128) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.processElement(MySqlRecordEmitter.java:105) at com.ververica.cdc.connectors.mysql.source.reader.MySqlPipelineRecordEmitter.processElement(MySqlPipelineRecordEmitter.java:119) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:82) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:55) at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:160) at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:419) at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) at java.lang.Thread.run(Thread.java:750) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: amount of AddColumnEvent is already existed at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.sendRequestToCoordinator(SchemaOperator.java:123) ... 30 more Caused by: java.lang.IllegalArgumentException: amount of AddColumnEvent is already existed at com.ververica.cdc.common.utils.SchemaUtils.applyAddColumnEvent(SchemaUtils.java:73) at com.ververica.cdc.common.utils.SchemaUtils.applySchemaChangeEvent(SchemaUtils.java:53) at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaManager.applySchemaChange(SchemaManager.java:113) at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistryRequestHandler.handleSchemaChangeRequest(SchemaRegistryRequestHandler.java:102) at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistry.handleCoordinationRequest(SchemaRegistry.java:157) at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.deliverCoordinationRequestToCoordinator(DefaultOperatorCoordinatorHandler.java:143) at org.apache.flink.runtime.scheduler.SchedulerBase.deliverCoordinationRequestToCoordinator(SchedulerBase.java:1070) at org.apache.flink.runtime.jobmaster.JobMaster.sendRequestToCoordinator(JobMaster.java:616) at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRpcInvocation$1(PekkoRpcActor.java:309) at org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcInvocation(PekkoRpcActor.java:307) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:222) at org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor.handleRpcMessage(FencedPekkoRpcActor.java:85) at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

What did you see instead?

Fix it

Anything else?

No response

Are you willing to submit a PR?

lvyanquan commented 11 months ago

Thank you for discovering this issue. I occasionally encounter it and will try to fix it.

fanqiejiang8 commented 10 months ago

@lvyanquan hello 这个bug有修复好了吗 在那个分支呢

fanqiejiang8 commented 10 months ago

@biyinggun 修改一下这个参数 pekko.ask.timeout: 120s

Jazzylol commented 10 months ago

@biyinggun 修改一下这个参数 pekko.ask.timeout: 120s 这个参数的作用是？

everhopingandwaiting commented 10 months ago

After compiling and using the latest code branch, I discovered that when I attempt to synchronize data starting from a specific binlog file, I still encounter an exception "AddColumnEvent is already existed," which leads to the termination of the task.

liisaxin commented 9 months ago

i discovered the same bug, too.

herunkang2018 commented 9 months ago

Hi @lvyanquan, could you have time to fix this issue? This is quite improtant for users who use schema sync feature.

liisaxin commented 9 months ago

Hi @lvyanquan, could you have time to fix this issue? This is quite improtant for users who use schema sync feature.

this is very very important . hope fix this bug quickly . @lvyanquan

sheltonsuen commented 8 months ago

After upstream add a new column, my flink cdc task keep trying to recover from the exception, but failed, seems like the same issue we have here. is there any suggestion to solve this? thanks

2024-03-05 19:25:58,688 INFO  com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistryRequestHandler [] - Received schema change event request from table newchiwan_trade.t_freight_source. Start to buffer requests for others.
2024-03-05 19:25:58,688 INFO  com.ververica.cdc.runtime.operators.schema.coordinator.SchemaManager [] - Handling schema change event: AddColumnEvent{tableId=newchiwan_trade.t_freight_source, addedColumns=[ColumnWithPosition{column=`channel` TINYINT, position=AFTER, existingColumn=`oil_fee_type` BIGINT}]}
2024-03-05 19:25:58,740 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Source: Flink CDC Event Source: mysql -> SchemaOperator -> PrePartition (1/1) (4e910aad380ccfd4548820d42d93e88d_cbc357ccb763df2852fee8c4fc7d55f2_0_154823) switched from RUNNING to FAILED on flink-cdc-dp-sr-cluster-taskmanager-1-3 @ 172.16.68.31 (dataPort=40224).
java.lang.IllegalStateException: Failed to send request to coordinator: com.ververica.cdc.runtime.operators.schema.event.SchemaChangeRequest@cfa0af2
    at com.ververica.cdc.runtime.operators.schema.SchemaOperator.sendRequestToCoordinator(SchemaOperator.java:126) ~[?:?]
    at com.ververica.cdc.runtime.operators.schema.SchemaOperator.requestSchemaChange(SchemaOperator.java:110) ~[?:?]
    at com.ververica.cdc.runtime.operators.schema.SchemaOperator.handleSchemaChangeEvent(SchemaOperator.java:95) ~[?:?]
    at com.ververica.cdc.runtime.operators.schema.SchemaOperator.processElement(SchemaOperator.java:85) ~[?:?]
    at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:309) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101) ~[flink-dist-1.18.1.jar:1.18.1]
    at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter$OutputCollector.collect(MySqlRecordEmitter.java:150) ~[?:?]
    at java.util.ArrayList.forEach(Unknown Source) ~[?:?]
    at com.ververica.cdc.debezium.event.DebeziumEventDeserializationSchema.deserialize(DebeziumEventDeserializationSchema.java:92) ~[?:?]
    at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitElement(MySqlRecordEmitter.java:128) ~[?:?]
    at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.processElement(MySqlRecordEmitter.java:105) ~[?:?]
    at com.ververica.cdc.connectors.mysql.source.reader.MySqlPipelineRecordEmitter.processElement(MySqlPipelineRecordEmitter.java:119) ~[?:?]
    at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:82) ~[?:?]
    at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:55) ~[?:?]
    at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:160) ~[flink-connector-files-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:419) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.18.1.jar:1.18.1]
    at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: channel of AddColumnEvent is already existed
    at java.util.concurrent.CompletableFuture.reportGet(Unknown Source) ~[?:?]
    at java.util.concurrent.CompletableFuture.get(Unknown Source) ~[?:?]
    at com.ververica.cdc.runtime.operators.schema.SchemaOperator.sendRequestToCoordinator(SchemaOperator.java:123) ~[?:?]
    ... 30 more
Caused by: java.lang.IllegalArgumentException: channel of AddColumnEvent is already existed
    at com.ververica.cdc.common.utils.SchemaUtils.applyAddColumnEvent(SchemaUtils.java:73) ~[?:?]
    at com.ververica.cdc.common.utils.SchemaUtils.applySchemaChangeEvent(SchemaUtils.java:53) ~[?:?]
    at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaManager.applySchemaChange(SchemaManager.java:113) ~[?:?]
    at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistryRequestHandler.handleSchemaChangeRequest(SchemaRegistryRequestHandler.java:102) ~[?:?]
    at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistry.handleCoordinationRequest(SchemaRegistry.java:157) ~[?:?]
    at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.deliverCoordinationRequestToCoordinator(DefaultOperatorCoordinatorHandler.java:143) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.runtime.scheduler.SchedulerBase.deliverCoordinationRequestToCoordinator(SchedulerBase.java:1070) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.runtime.jobmaster.JobMaster.sendRequestToCoordinator(JobMaster.java:616) ~[flink-dist-1.18.1.jar:1.18.1]
    at jdk.internal.reflect.GeneratedMethodAccessor63.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
    at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRpcInvocation$1(PekkoRpcActor.java:309) ~[flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[flink-dist-1.18.1.jar:1.18.1]
    at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcInvocation(PekkoRpcActor.java:307) ~[flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:222) ~[flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor.handleRpcMessage(FencedPekkoRpcActor.java:85) ~[flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168) ~[flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) [flink-rpc-akkae9f6bb37-e207-4e36-999a-f42a36fa8818.jar:1.18.1]
    at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?]
    at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?]
    at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?]
    at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?]
    at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]
2024-03-05 19:25:58,742 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Removing registered reader after failure for subtask 0 (#154823) of source Source: Flink CDC Event Source: mysql.
2024-03-05 19:25:58,742 INFO  org.apache.flink.runtime.jobmaster.JobMaster                 [] - 2 tasks will be restarted to recover the failed task 4e910aad380ccfd4548820d42d93e88d_cbc357ccb763df2852fee8c4fc7d55f2_0_154823.
2024-03-05 19:25:58,742 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Job Sync newchiwan Databases to StarRocks (41d15c80977ce8df2c3c5609c41e4554) switched from state RUNNING to RESTARTING.
2024-03-05 19:25:58,742 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - PostPartition -> Sink Writer: newchiwan_cdc -> Sink Committer: newchiwan_cdc (1/1) (4e910aad380ccfd4548820d42d93e88d_0deb1b26a3d9eb3c8f0c11f7110b2903_0_154823) switched from RUNNING to CANCELING.
2024-03-05 19:25:58,772 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - PostPartition -> Sink Writer: newchiwan_cdc -> Sink Committer: newchiwan_cdc (1/1) (4e910aad380ccfd4548820d42d93e88d_0deb1b26a3d9eb3c8f0c11f7110b2903_0_154823) switched from CANCELING to CANCELED.
2024-03-05 19:25:58,772 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager [] - Clearing resource requirements of job 41d15c80977ce8df2c3c5609c41e4554
2024-03-05 19:25:58,772 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedTaskManagerTracker [] - Clear all pending allocations for job 41d15c80977ce8df2c3c5609c41e4554.
2024-03-05 19:25:59,509 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Triggering checkpoint 147500 (type=CheckpointType{name='Checkpoint', sharingFilesStrategy=FORWARD_BACKWARD}) @ 1709637959509 for job 0124a1157b4545b0e88933f591baa153.
2024-03-05 19:25:59,688 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Completed checkpoint 147500 for job 0124a1157b4545b0e88933f591baa153 (465658 bytes, checkpointDuration=179 ms, finalizationTime=0 ms).
2024-03-05 19:25:59,688 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Marking checkpoint 147500 as completed for source Source: Flink CDC Event Source: mysql.
2024-03-05 19:25:59,742 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Job Sync newchiwan Databases to StarRocks (41d15c80977ce8df2c3c5609c41e4554) switched from state RESTARTING to RUNNING.
2024-03-05 19:25:59,743 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Restoring job 41d15c80977ce8df2c3c5609c41e4554 from Checkpoint 49479 @ 1709637952671 for 41d15c80977ce8df2c3c5609c41e4554 located at file:/opt/storage/checkpoints/41d15c80977ce8df2c3c5609c41e4554/chk-49479.
2024-03-05 19:25:59,743 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - No master state to restore
2024-03-05 19:25:59,743 ERROR com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistry [] - Subtask 0 reset at checkpoint 49479.
PatrickRen commented 7 months ago

Closing this issue as it has been migrated to Apache Jira.

bimtauer commented 7 months ago

Is there a public Jira tracker to follow along? Just tried running the quickstart tutorial and encountered the same error upon adding a column.

PatrickRen commented 7 months ago

@bimtauer The issue has been migrated to https://issues.apache.org/jira/browse/FLINK-34849. Let's continue the discussion there.