search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
36.88k stars 5.81k forks source link

add index failed with error “Error 1105 (HY000): encountered TSO stream being closed unexpectedly” or "Error 1105 (HY000): create TSO stream failed, retry timeout" when kill pd leader #52490

Open Lily2025 opened 5 months ago

Lily2025 commented 5 months ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1、run sysbench 2、add index for one of table 3、kill pd leader

tidb logs: tidb-0.tar.gz tidb-1.tar.gz

2. What did you expect to see? (Required)

add index can success

3. What did you see instead (Required)

add index failed with error “Error 1105 (HY000): encountered TSO stream being closed unexpectedly”

add index failed at 2024-04-10 19:52:00: Error 1105 (HY000): encountered TSO stream being closed unexpectedly operatorLogs: [2024-04-10 19:48:55] ###### start adding index ALTER TABLE sbtest1 ADD INDEX index_test_1712749735022(c) [2024-04-10 19:48:55] ###### wait for ddl job finish

4. What is your TiDB version? (Required)

./tidb-server -V Release Version: v8.1.0-alpha Edition: Community Git Commit Hash: 33f5d05c478f18598b1a4b187a70ef1a54d896d4 Git Branch: heads/refs/tags/v8.1.0-alpha UTC Build Time: 2024-04-08 11:45:05 GoVersion: go1.21.6 Race Enabled: false Check Table Before Drop: false Store: unistore 2024-04-10T08:51:01.611+0800

Lily2025 commented 5 months ago

/assign ywqzzy /type bug /severity major

Lily2025 commented 4 months ago

/assign JmPotato

Lily2025 commented 4 months ago

from @JmPotato https://github.com/tikv/pd/pull/8123 can reduce the probability of problems

Lily2025 commented 1 month ago

another issue: kill pd leader version: 8.3.0

add index failed at 2024-07-29 08:04:15: Error 1105 (HY000): create TSO stream failed, retry timeout operatorLogs: [2024-07-29 08:00:20] ###### start adding index ALTER TABLE sbtest1 ADD INDEX index_test_1722211220993(c) [2024-07-29 08:00:20] ###### wait for ddl job finish

tidb logs: tidb-0-52490.log