apache / iceberg

Apache Iceberg
https://iceberg.apache.org/
Apache License 2.0
6.49k stars 2.24k forks source link

When `distribution-mode` = RANGE, Flink Stop-with-savepoint operation will definitely fail #11556

Open huyuanfeng2018 opened 1 week ago

huyuanfeng2018 commented 1 week ago

Apache Iceberg version

1.7.0 (latest release)

Query engine

Flink

Please describe the bug 🐞

When my job turns on distribution-mode = RANGE, and I try to stop the task and generate Savepoint at the same time, an exception will occur.

The key case is:

The vertex Source:xxx[10] -> Calc[11] -> range-shuffle (id = xxxxx) has used UnionListState, but part of its tasks has called operators' finish method.

Willingness to contribute