doris flink connector do not support flink 1.13 [Bug]

xhmz commented 3 years ago

Search before asking

[X] I had searched in the issues and found no similar issues.

Version

1.13

What's Wrong?

org.apache.flink.table.data.binary.BinaryRowData cannot be cast to org.apache.flink.table.data.GenericRowData at org.apache.doris.flink.table.DorisDynamicOutputFormat.addBatch(DorisDynamicOutputFormat.java:142) at org.apache.doris.flink.table.DorisDynamicOutputFormat.writeRecord(DorisDynamicOutputFormat.java:133) at org.apache.doris.flink.table.DorisDynamicOutputFormat.writeRecord(DorisDynamicOutputFormat.java:45) at org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.invoke(OutputFormatSinkFunction.java:87) at org.apache.flink.streaming.api.functions.sink.SinkFunction.invoke(SinkFunction.java:49) at org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:65) at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:205) at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134) at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:66) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:423) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:204) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:681) at org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:636) at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:647) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:620) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566) at java.lang.Thread.run(Thread.java:748)

What You Expected?

no error

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

JNSimba commented 3 years ago

can you show your flinksql ?

xhmz commented 3 years ago

doris sink format， not use GenericRowData. use RowData::FieldGetter， fixed it。

xhmz commented 3 years ago

https://github.com/apache/incubator-doris/pull/6655

xhmz commented 3 years ago

set pipeline.operator-chaining = false
this param cause the bug

JNSimba commented 3 years ago

doris sink format， not use GenericRowData. use RowData::FieldGetter， fixed it。

great，use RowData::FieldGetter is worked

JNSimba commented 3 years ago

set pipeline.operator-chaining = false this param cause the bug

I try to set disableOperatorChaining , this problem still occurs . such as flinksql has limit xxxx . and this parameter may be reduces flink performance

xhmz commented 3 years ago

yes， if you disableOperatorChaining， RowDataSerializer will Serialize RowData to BinaryRowData

xhmz commented 3 years ago

set pipeline.operator-chaining = false this param cause the bug

I try to set disableOperatorChaining , this problem still occurs . such as flinksql has limit xxxx . and this parameter may be reduces flink performance

我的意思是设置这个参数会导致这个错误～

xhmz commented 3 years ago

https://github.com/apache/incubator-doris/pull/6655

apache / doris