Rows are lost for 100K table and go-ycsb workload test

[Tammyxia]

What did you do?

• BR restore 100K source data to upstream and downstream TiDB cluster.
• Create a changefeed, then run workload: go-ycsb load and run on table prefix unit2_.
• Run test case with chaos: upstream all node restart, cdc owner switch, all upstream unavailable, PD leader switch.
• Check data correctness between upstream and downstream TiDB cluster.

What did you expect to see?

Data consistency

What did you see instead?

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

Upstream TiKV version (execute tikv-server --version):

TiKV
Release Version:   6.0.0-alpha
Edition:           Community
Git Commit Hash:   ac32d5c8099a85994a9790ae1840755033dd0fac
Git Commit Branch: heads/refs/tags/v6.1.0-nightly
UTC Build Time:    2022-03-21 23:16:34
Rust Version:      rustc 1.60.0-nightly (1e12aef3f 2022-02-13)
Enable Features:   jemalloc mem-profiling portable sse test-engines-rocksdb cloud-aws cloud-gcp cloud-azure
Profile:           dist_release

TiCDC version (execute cdc version):

(paste TiCDC version here)
elease Version: v5.4.0-master
Git Commit Hash: be5524dd90d5db245a22cc125599b5c81d78c2bb
Git Branch: heads/refs/tags/v6.1.0-nightly
UTC Build Time: 2022-03-21 23:15:49
Go Version: go version go1.18 linux/amd64
Failpoint Build: false

[Tammyxia]

Reproduce again in this test.

[overvenus]

It should be fixed by https://github.com/tikv/tikv/pull/12262 let's keep it open and wait for another round of testing.

[overvenus]

Can't reproduce again.