apache / flink-cdc

Flink CDC is a streaming data integration tool
https://nightlies.apache.org/flink/flink-cdc-docs-stable
Apache License 2.0
5.7k stars 1.94k forks source link

NumberFormatException: For input string: "0.00" #2200

Closed zhengcongyi closed 7 months ago

zhengcongyi commented 1 year ago

Search before asking

Flink version

flink 1.13.2

Flink CDC version

2.2.1

Database and its version

polardb mysql 5.7

Minimal reproduce step

Caused by: java.lang.NumberFormatException: For input string: "0.00" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_151] at java.lang.Integer.parseInt(Integer.java:580) ~[?:1.8.0_151] at java.lang.Integer.valueOf(Integer.java:766) ~[?:1.8.0_151] at io.debezium.jdbc.JdbcValueConverters.lambda$convertInteger$41(JdbcValueConverters.java:892) ~[debezium-core-1.5.4.Final.jar:1.5.4.Final] at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1288) ~[debezium-core-1.5.4.Final.jar:1.5.4.Final] at io.debezium.jdbc.JdbcValueConverters.convertInteger(JdbcValueConverters.java:880) ~[debezium-core-1.5.4.Final.jar:1.5.4.Final] at io.debezium.jdbc.JdbcValueConverters.lambda$converter$5(JdbcValueConverters.java:295) ~[debezium-core-1.5.4.Final.jar:1.5.4.Final] at io.debezium.connector.mysql.MySqlDefaultValueConverter.setColumnDefaultValue(MySqlDefaultValueConverter.java:405) ~[debezium-connector-mysql-1.5.4.Final.jar:1.5.4.Final] at io.debezium.connector.mysql.antlr.listener.CreateTableParserListener.convertDefaultValueToSchemaType(CreateTableParserListener.java:147) ~[debezium-connector-mysql-1.5.4.Final.jar:1.5.4.Final] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_151] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_151] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1380) ~[?:1.8.0_151] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_151] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_151] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_151] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_151] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_151] at io.debezium.connector.mysql.antlr.listener.CreateTableParserListener.lambda$exitColumnCreateTable$1(CreateTableParserListener.java:75) ~[debezium-connector-mysql-1.5.4.Final.jar:1.5.4.Final] at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.runIfNotNull(MySqlAntlrDdlParser.java:357) ~[debezium-connector-mysql-1.5.4.Final.jar:1.5.4.Final] at io.debezium.connector.mysql.antlr.listener.CreateTableParserListener.exitColumnCreateTable(CreateTableParserListener.java:55) ~[debezium-connector-mysql-1.5.4.Final.jar:1.5.4.Final] at io.debezium.ddl.parser.mysql.generated.MySqlParser$ColumnCreateTableContext.exitRule(MySqlParser.java:4733) ~[debezium-ddl-parser-1.5.4.Final.jar:1.5.4.Final] at io.debezium.antlr.ProxyParseTreeListenerUtil.delegateExitRule(ProxyParseTreeListenerUtil.java:64) ~[debezium-ddl-parser-1.5.4.Final.jar:1.5.4.Final] at io.debezium.connector.mysql.antlr.listener.MySqlAntlrDdlParserListener.exitEveryRule(MySqlAntlrDdlParserListener.java:106) ~[debezium-connector-mysql-1.5.4.Final.jar:1.5.4.Final] at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:48) ~[antlr4-runtime-4.7.2.jar:4.7.2] at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:30) ~[antlr4-runtime-4.7.2.jar:4.7.2] at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)

新增字段为decimal default '0.0' 已经在同步任务就挂了,报这个错误

What did you expect to see?

1

What did you see instead?

1

Anything else?

No response

Are you willing to submit a PR?

tuoluzhe8521 commented 1 year ago

have you solve the problem,bro?

zhengcongyi commented 1 year ago

还没解决,库某张表新增一个字段为decimal default '0.0' ,影响到已经在跑同步任务,都会报For input string: "0.00"这个,然后全量同步或者最后同步,就正常了。已经碰到两次了。一直没解决

tuoluzhe8521 commented 1 year ago

but you did not start the failed task with a checkppoint?

tuoluzhe8521 commented 1 year ago

还没解决,库某张表新增一个字段为decimal default '0.0' ,影响到已经在跑同步任务,都会报For input string: "0.00"这个,然后全量同步或者最后同步,就正常了。已经碰到两次了。一直没解决

i had some test, and it seems that it is caused by the int type,when i add a column and defind it as an int type,but i set the default value as a double one, when the program caught the change,then the error comes

zhengcongyi commented 1 year ago

but you did not start the failed task with a checkppoint?

有,就是启不起来,不能从checkpoint恢复

zhengcongyi commented 1 year ago

还没解决,库某张表新增一个字段为decimal default '0.0' ,影响到已经在跑同步任务,都会报For input string: "0.00"这个,然后全量同步或者最后同步,就正常了。已经碰到两次了。一直没解决

i had some test, and it seems that it is caused by the int type,when i add a column and defind it as an int type,but i set the default value as a double one, when the program caught the change,then the error comes

我看mysql脚本,都是新增字段,而且修改表,会影响到其它表同步任务,也是报这个错误

zhengcongyi commented 1 year ago
image

columnEditor.create() jdbcType默认是4,JdbcValueConverters 这边转换对应的是INT,是不是有问题?

tuoluzhe8521 commented 1 year ago

that 's because you add an illegal default value to an int value,so the error happed. image it just accepts an int value,and then cast it to an int

zhengcongyi commented 1 year ago

illegal default 我搜索一下数据库,看没有默认值为0.00的int,而且已经在同步的表没新增字段,也会报0.00

tuoluzhe8521 commented 1 year ago

are you sure everyone did not add any colunm? the default value 0.0 has been fotmat to 0, you can have a test too,now as i know,when the value type is a integer,but you set the default value as a double,then the binlog will send 0.0 to the program,but the data change will work well and it will just send a integer to the program,i guess mysql have change it to a legal value then write it to the binlog

zhengcongyi commented 1 year ago

are you sure everyone did not add any colunm? the default value 0.0 has been fotmat to 0, you can have a test too,now as i know,when the value type is a integer,but you set the default value as a double,then the binlog will send 0.0 to the program,but the data change will work well and it will just send a integer to the program,i guess mysql have change it to a legal value then write it to the binlog

跟mysql db 盘查了上线sql脚本,没有只修改字段默认值,只有新增字段

zhengcongyi commented 1 year ago

are you sure everyone did not add any colunm? the default value 0.0 has been fotmat to 0, you can have a test too,now as i know,when the value type is a integer,but you set the default value as a double,then the binlog will send 0.0 to the program,but the data change will work well and it will just send a integer to the program,i guess mysql have change it to a legal value then write it to the binlog

查看sql只有 ADD COLUMN 。还有没有其它问题导致的

tuoluzhe8521 commented 1 year ago

the problem that cause the error is adding the column,someone add columns which is int type,but define the default value as an double one,you can check the sql and you will find this problem

tuoluzhe8521 commented 1 year ago

or you can have a test, add a int type column and define the default value as a double, then you will reproduce the error 100%

PatrickRen commented 7 months ago

Considering collaboration with developers around the world, please re-create your issue in English on Apache Jira under project Flink with component tag Flink CDC. Thank you!