pingcap / tidb

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.
https://pingcap.com
Apache License 2.0
37.41k stars 5.85k forks source link

The timeout is too short for fetching data of `INFORMATION_SCHEMA.TIFLASH_TABLES` #57816

Open JaySon-Huang opened 1 day ago

JaySon-Huang commented 1 day ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

An issue reported in the asktug (https://asktug.com/t/topic/1036157/1)

mysql> select * from INFORMATION_SCHEMA.TIFLASH_TABLES  limit 1\G
ERROR 1105 (HY000): rpc error: code = DeadlineExceeded desc = context deadline exceeded

2. What did you expect to see? (Required)

3. What did you see instead (Required)

The timeout is only 1 second for fetching the system table from tiflash instance. When there are thousand of table with tiflash replica, the query easily run into timeout.

https://github.com/pingcap/tidb/blob/v7.5.4/pkg/executor/infoschema_reader.go#L3081-L3083

There are about 5000 tables/partition in the cluster.

mysql> select count(*) from INFORMATION_SCHEMA.TIFLASH_REPLICA;
+----------+
| count(*) |
+----------+
|     3090 |
+----------+
1 row in set (0.04 sec)

mysql> select count(*) from INFORMATION_SCHEMA.TIFLASH_REPLICA t join INFORMATION_SCHEMA.partitions p on t.table_schema = p.table_schema and t.table_name= p.table_name;
+----------+
| count(*) |
+----------+
|     4892 |
+----------+
1 row in set (0.30 sec)

4. What is your TiDB version? (Required)

v7.5.4

JaySon-Huang commented 1 day ago

/label affects-8.5

JaySon-Huang commented 1 day ago

/label affects-8.1

JaySon-Huang commented 1 day ago

/label affects-7.5

JaySon-Huang commented 1 day ago

/label affects-7.1

JaySon-Huang commented 1 day ago

/label affects-6.5