changefeed lag reached 458s when inject one of az failure

[Lily2025]

What did you do?

1、create changefeed 2、run sysbench 3、inject one of az failure last for 10mins

What did you expect to see?

changefeed lag less than 120s during failure

What did you see instead?

changefeed lag reached 458s when inject one of az failure

Versions of the cluster

./cdc version Release Version: v8.2.0-alpha Git Commit Hash: 4d89c24642aaec2faa46c3e98bfa77c87151ae36 Git Branch: heads/refs/tags/v8.2.0-alpha UTC Build Time: 2024-05-21 11:40:40 Go Version: go version go1.21.10 linux/amd64 Failpoint Build: false 2024-05-22T15:42:57.998+0800

./tikv-server -V TiKV Release Version: 8.2.0-alpha Edition: Community Git Commit Hash: 34d8b38a17e3003cc5730e8dc53d447db5b7bd31 Git Commit Branch: heads/refs/tags/v8.2.0-alpha UTC Build Time: 2024-05-20 11:38:26 Rust Version: rustc 1.77.0-nightly (89e2160c4 2023-12-27) Enable Features: memory-engine pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine trace-async-tasks openssl-vendored Profile: dist_release 2024-05-22T15:42:58.797+0800

current status of DM cluster (execute query-status <task-name> in dmctl)

No response

[Lily2025]

/remove-area dm /area ticdc

[fubinzh]

/severity moderate