search

apache / inlong

Apache InLong - a one-stop, full-scenario integration framework for massive data
https://inlong.apache.org/
Apache License 2.0
1.4k stars 530 forks source link

【咨询】iceberg sink新增列 #10699

Closed tivontang closed 3 months ago

tivontang commented 4 months ago

flink iceberg sink, 如果sink table运营中新增列，是否需要重启任务，如果不重启，项目如何实现处理该场景

github-actions[bot] commented 4 months ago

Hello @tivontang, thank you for opening your first issue in InLong 🧡 We will respond as soon as possible ⏳ If this is a bug report, please provide screenshots or error logs for us to reproduce your issue, so we can do our best to fix it. If you have any questions in the meantime, you can also ask us on the InLong Discussions 🔍

vernedeng commented 4 months ago

flink iceberg sink, 如果sink table运营中新增列，是否需要重启任务，如果不重启，项目如何实现处理该场景

The task must be restarted at the moment. This issue can actually be resolved by adding a new operator, which acquires the latest table schema at each checkpoint and updates the other operators.

vernedeng commented 4 months ago

flink iceberg sink, 如果sink table运营中新增列，是否需要重启任务，如果不重启，项目如何实现处理该场景

better to write an issues in English.