pingcap / dumpling

Dumpling is a fast, easy-to-use tool written by Go for dumping data from the database(MySQL, TiDB...) to local/cloud(S3, GCP...) in multifarious formats(SQL, CSV...).
Apache License 2.0
281 stars 85 forks source link

Dumpling should report error if --consistency is lock and --snapshot parameter is provided #370

Closed fubinzh closed 2 years ago

fubinzh commented 2 years ago

What did you do?

Run dumpling, specify --consistency as lock, and specify --snapshot as well.

What did you expect to see?

When consistency is lock, --snapshot parameters should be ignored, error should be prompted appropriately.

What did you see instead?

Dumpling continues, and the error message is wrong and confusing.

[root@IDC-172-16-6-35 v5.0.0-nightly-20211010]# ./dumpling -u root -h 172.16.6.35 -P4000 --filetype csv -F 256MiB -o 's3://nfs/tmpxxx' --database sbtest --consistency lock --snapshot "2021-10-12 16:16:28" Release version: v5.3.0-alpha-3-g562cb6c Git commit hash: 562cb6c021782ce3e35c279a611e7a381979acfd Git branch: master Build timestamp: 2021-10-09 08:20:31Z Go version: go version go1.16.4 linux/amd64

[2021/10/12 16:16:53.889 +08:00] [INFO] [versions.go:55] ["Welcome to dumpling"] ["Release Version"=v5.3.0-alpha-3-g562cb6c] ["Git Commit Hash"=562cb6c021782ce3e35c279a611e7a381979acfd] ["Git Branch"=master] ["Build timestamp"="2021-10-09 08:20:31"] ["Go Version"="go version go1.16.4 linux/amd64"] [2021/10/12 16:16:53.890 +08:00] [INFO] [config.go:608] ["detect server type"] [type=TiDB] [2021/10/12 16:16:53.890 +08:00] [INFO] [config.go:627] ["detect server version"] [version=5.3.0-alpha-181-g1e8b9337e] [2021/10/12 16:16:53.894 +08:00] [INFO] [client.go:215] ["[pd] create pd client with endpoints"] [pd-address="[172.16.6.35:2379]"] [2021/10/12 16:16:53.895 +08:00] [INFO] [base_client.go:364] ["[pd] switch leader"] [new-leader=http://172.16.6.35:2379] [old-leader=] [2021/10/12 16:16:53.895 +08:00] [INFO] [base_client.go:127] ["[pd] init cluster id"] [cluster-id=7003510880105342509] [2021/10/12 16:16:53.895 +08:00] [INFO] [client.go:239] ["[pd] create tso dispatcher"] [dc-location=global] [2021/10/12 16:16:53.895 +08:00] [INFO] [dump.go:1151] ["generate dumpling gc safePoint id"] [id=dumpling_1634026613895981022] [2021/10/12 16:16:53.895 +08:00] [INFO] [dump.go:93] ["begin to run Dump"] [conf="{"s3":{"endpoint":"http://minio.pingcap.net:9000","region":"us-east-1","storage-class":"","sse":"","sse-kms-key-id":"","acl":"","access-key":"minioadmin","secret-access-key":"minioadmin","provider":"","force-path-style":true,"use-accelerate-endpoint":false},"gcs":{"endpoint":"","storage-class":"","predefined-acl":"","credentials-file":""},"AllowCleartextPasswords":false,"SortByPk":true,"NoViews":true,"NoHeader":false,"NoSchemas":false,"NoData":false,"CompleteInsert":false,"TransactionalConsistency":true,"EscapeBackslash":true,"DumpEmptyDatabase":true,"PosAfterConnect":false,"CompressType":0,"Host":"172.16.6.35","Port":4000,"Threads":4,"User":"root","Security":{"CAPath":"","CertPath":"","KeyPath":""},"LogLevel":"info","LogFile":"","LogFormat":"text","OutputDirPath":"s3://nfs/tmp?access-key=minioadmin\u0026secret-access-key=minioadmin\u0026endpoint=http%3a%2f%2fminio.pingcap.net%3a9000\u0026force-path-style=true","StatusAddr":":8281","Snapshot":"2021-10-12 16:16:28","Consistency":"lock","CsvNullValue":"\N","SQL":"","CsvSeparator":",","CsvDelimiter":"\"","Databases":["sbtest"],"Where":"","FileType":"csv","ServerInfo":{"HasTiKV":false,"ServerType":3,"ServerVersion":"5.3.0-alpha-181-g1e8b9337e"},"Rows":0,"ReadTimeout":900000000000,"TiDBMemQuotaQuery":0,"FileSize":268435456,"StatementSize":1000000,"SessionParams":{},"Tables":null}"] [2021/10/12 16:16:53.897 +08:00] [INFO] [client.go:511] ["[pd] exit tso dispatcher"] [dc-location=global] [2021/10/12 16:16:53.897 +08:00] [ERROR] [main.go:77] ["dump failed error stack info"] [error="Unknown databases [sbtest]"] [errorVerbose="Unknown databases

Versions of the cluster

Dumpling:v5.3.0-alpha-3-g562cb6c

fubinzh commented 2 years ago

/severity major

lichunzhu commented 2 years ago

/cc @docsir

fubinzh commented 2 years ago

Similar issue exists when specify consistency:none and snapshot together.