pingcap / tiflow

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
Apache License 2.0
430 stars 286 forks source link

changefeed failed after running sysbench prepare with simple protocol #10754

Closed Lily2025 closed 8 months ago

Lily2025 commented 8 months ago

What did you do?

1、create changefeed with simple protocol sink_url_options: simple-test?protocol=simple&max-message-bytes=1024000&partition-num=3&replication-factor=3 config: "[sink]\nsend-bootstrap-interval-in-sec = 120\nsend-bootstrap-in-msg-count = 10000\n[sink.kafka-config.codec-config]\nencoding-format = \"avro\"\n[sink.kafka-config.large-message-handle]\nlarge-message-handle-compression = \"lz4\"\nlarge-message-handle-option = \"claim-check\"\nclaim-check-storage-uri = \"s3://tmp?access-key=minioadmin&secret-access-key=minioadmin&endpoint=http%3a%2f%2fminio-peer%3a9000\""

2、run sysbench prepare sysbench --db-driver=mysql --time=0 --mysql-db=sysbench_32_100w --mysql-host=tc-tidb.testbed-glh-cdc-w2sts --mysql-port=4000 --threads=32 --tables=32 --table-size=1000000 --mysql-ignore-errors=\"all\" --mysql-user=root --report-interval=10 oltp_read_only prepare

ticdc-0.tar.gz

What did you expect to see?

changefeed is normal

What did you see instead?

changefeed failed after running sysbench prepare with simple protocol

{ "upstream_id": 7343517049376113793, "namespace": "default", "id": "simple", "state": "failed", "checkpoint_tso": 448301266712920067, "checkpoint_time": "2024-03-11 05:06:56.306", "error": { "time": "2024-03-11T06:56:30.105966075Z", "addr": "tc-ticdc-0.tc-ticdc-peer.testbed-glh-cdc-w2sts.svc:8301", "code": "CDC:ErrChangefeedUnretryable", "message": "table sink stuck" } }

Versions of the cluster

./cdc version Release Version: v8.0.0-alpha Git Commit Hash: 8c51dfa5c08543395e564ae526478990105e259b Git Branch: heads/refs/tags/v8.0.0-alpha UTC Build Time: 2024-03-06 11:37:40 Go Version: go version go1.21.6 linux/amd64 Failpoint Build: false

current status of DM cluster (execute query-status <task-name> in dmctl)

No response

Lily2025 commented 8 months ago

/remove-area dm /area ticdc /assign CharlesCheung96

Lily2025 commented 8 months ago

/type bug /severity major

CharlesCheung96 commented 8 months ago

The root cause is #10739