apache / doris-flink-connector

Flink Connector for Apache Doris
https://doris.apache.org/
Apache License 2.0
292 stars 201 forks source link

[Bug] [Source]DorisSplitRecords.finishedSplits()返回的集合有null对象导致的整个flink作业发生NullPointerException #411

Open yingh0ng opened 1 week ago

yingh0ng commented 1 week ago

Search before asking

Version

1.6.0

What's Wrong?

DorisSplitRecords.finishSplits()返回的集合里有且仅有一个null对象 1、第一个图里添加分片后是正常的; 2、第二个图的日志显示FetcheTask里的callBack打印出来的finishSplits里仅有一个null对象 企业微信截图_17194749296203

企业微信截图_17194741888949

What You Expected?

正常执行

How to Reproduce?

使用doris-flink-connector读取doris的表并写到另一个doris的表里

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

JNSimba commented 1 week ago

Please post the error stack

yingh0ng commented 1 week ago

Please post the error stack

npe.log

yingh0ng commented 1 week ago

@JNSimba Is it possible that there was a reordering of instructions in this place? Because String is immutable, and the previous log has printed split information. And this problem happens occasionally. image

JNSimba commented 1 week ago

@yingh0ng Thank you, could you please post the table schema and DorisSource configuration?

JNSimba commented 1 week ago

@yingh0ng Can you try this PR? It fixes the problem that the SplitId of DorisSource is repeated. #414

yingh0ng commented 6 days ago

@yingh0ng Thank you, could you please post the table schema and DorisSource configuration?

Only have table properties of DDL:

table:
    properties:
      replication_allocation: 'tag.location.default: 1'
      enable_duplicate_without_keys_by_default: true

The DorisSource configuration:

connector=doris
fenodes=xxx
table.identifier=database.tableName
username=xxx
password=xxx
doris.request.tablet.size=10(my table`s buckets is 10)
yingh0ng commented 6 days ago

@yingh0ng Can you try this PR? It fixes the problem that the SplitId of DorisSource is repeated. #414

Got it. Thank you!