pingcap / dumpling

Dumpling is a fast, easy-to-use tool written by Go for dumping data from the database(MySQL, TiDB...) to local/cloud(S3, GCP...) in multifarious formats(SQL, CSV...).
Apache License 2.0
280 stars 85 forks source link

dumpling doesn't get the primary key columns correctly #310

Closed lichunzhu closed 3 years ago

lichunzhu commented 3 years ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a recipe for reproducing the error. Use dumpling to dump a table which a int column with both primary key and unique key. But dumpling didn't return the right primary columns which caused dumpling failed to run selectFromTableRegion nor selectFromTableSample and failed the whole dump progress.

The following SQL is pasted from https://github.com/pingcap/dumpling/blob/v5.0.3/v4/export/sql.go#L326

mysql> select version();
+---------------------+
| version()           |
+---------------------+
| 5.7.25-TiDB-v3.0.17 |
+---------------------+
1 row in set (0.00 sec)

mysql> SELECT c.COLUMN_NAME, DATA_TYPE FROM
    -> INFORMATION_SCHEMA.COLUMNS c INNER JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE k ON
    -> c.column_name = k.column_name and
    -> c.table_schema = k.table_schema and
    -> c.table_name = k.table_name and
    -> c.table_schema = 'test' and
    -> c.table_name = 't9'
    -> WHERE COLUMN_KEY = 'PRI'
    -> ORDER BY k.ORDINAL_POSITION;
+-------------+-----------+
| COLUMN_NAME | DATA_TYPE |
+-------------+-----------+
| a           | int       |
| a           | int       |
+-------------+-----------+
2 rows in set (0.03 sec)

mysql> show create table t9;
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                        |
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| t9    | CREATE TABLE `t9` (
  `a` int(11) NOT NULL,
  PRIMARY KEY (`a`),
  UNIQUE KEY `a` (`a`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
  1. What did you expect to see? Dumpling dumps the data correctly.

  2. What did you see instead?

    
    [2021/07/12 23:58:49.230 +08:00] [ERROR] [main.go:77] ["dump failed error stack info"] [error="unsupported primary key for selectTableRegion. pkFields: [id, id], pkColTypes: [BIGINT, BIGINT]"] [errorVerbose="unsupported primary key for selectTableRegion. pkFields: [id, id], pkColTypes: [BIGINT, BIGINT]\ngithub.com/pingcap/dumpling/v4/export.checkTiDBTableRegionPkFields\n\tgithub.com/pingcap/dumpling/v4/export/dump.go:732\ngithub.com/pingcap/dumpling/v4/export.selectTiDBRowKeyFields\n\tgithub.com/pingcap/dumpling/v4/export/dump.go:723\ngithub.com/pingcap/dumpling/v4/export.selectTiDBTableRegion\n\tgithub.com/pingcap/dumpling/v4/export/dump.go:742\ngithub.com/pingcap/dumpling/v4/export.(*Dumper).concurrentDumpTiDBTables\n\tgithub.com/pingcap/dumpling/v4/export/dump.go:592\ngithub.com/pingcap/dumpling/v4/export.(*Dumper).concurrentDumpTable\n\tgithub.com/pingcap/dumpling/v4/export/dump.go:446\ngithub.com/pingcap/dumpling/v4/export.(*Dumper).buildConcatTask.func1\n\tgithub.com/pingcap/dumpling/v4/export/dump.go:347\nruntime.goexit\n\truntime/asm_amd64.s:1371"]

dump failed: unsupported primary key for selectTableRegion. pkFields: [id, id], pkColTypes: [BIGINT, BIGINT]



4. Affected versions
TiDB v4.0.\* -> dumpling: v4.0.13, v5.0.2 ~ v5.0.3, v5.1.0
For TiDB v5.0.* dumpling will generate redundant where clauses. But this won't affect the result.