search

pingcap / dumpling

Dumpling is a fast, easy-to-use tool written by Go for dumping data from the database(MySQL, TiDB...) to local/cloud(S3, GCP...) in multifarious formats(SQL, CSV...).
Apache License 2.0
280 stars 85 forks source link

*: reduce dumpling accessing database and information_schema usage to improve its stability (#305) #314

Closed ti-chi-bot closed 3 years ago

ti-chi-bot commented 3 years ago

This is an automated cherry-pick of #305

What problem does this PR solve?

fix https://github.com/pingcap/dumpling/issues/298 and fix https://github.com/pingcap/dumpling/issues/296 and fix https://github.com/pingcap/dumpling/issues/310

What is changed and how it works?

  1. Try to dump the whole table directly if we meet some errors when we try to split tables into several chunks. https://github.com/pingcap/dumpling/commit/cfb258a98c791d07dab99e2f00519de23a375e1e

  2. Reduce accessing SelectedField and SelectedLen info from database https://github.com/pingcap/dumpling/commit/d3ba1e8d7282a823b3b5548acd31eb0dc3081b19

  3. Remove the usage of information_schema https://github.com/pingcap/dumpling/commit/1a0d9c4ecf46208ce0c8e88e19a5f4ae77f3a567, https://github.com/pingcap/dumpling/commit/7f7cf586cf99d5750c22834958cc17bd3b85b88e, https://github.com/pingcap/dumpling/commit/b333beaeb3ea898605bd2da5d9a11783eff4491e, https://github.com/pingcap/dumpling/commit/088b5ba98d3957369aa93856a3bc964960ced854, https://github.com/pingcap/dumpling/commit/7dac228759502c8ec79263968457e75f61bae381, https://github.com/pingcap/dumpling/commit/bdb3c26f4218cacbfc28c055c9e62c47cee784a6

  4. Fix the bug that dumpling finishedRows is not accurate if we retry in chunks. https://github.com/pingcap/dumpling/commit/ff398a48a7800971011cb1ec476ecb7196676271

  5. Fix the bug that dumpling dumps unlocked tables for consistency lock https://github.com/pingcap/dumpling/commit/7f7cf586cf99d5750c22834958cc17bd3b85b88e

  6. Fix the bug of no-schema argument's wrong output https://github.com/pingcap/dumpling/commit/09d257dc54faaf36e1beb7079f5592cfdac9d117

Check List

Tests

Side effects

Related changes

Release note

ti-chi-bot commented 3 years ago

[REVIEW NOTIFICATION]

This pull request has been approved by:

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment. After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review. Reviewer can cancel approval by submitting a request changes review.
lichunzhu commented 3 years ago

/merge

ti-chi-bot commented 3 years ago

This pull request has been accepted and is ready to merge.

Commit hash: a26447347cbc8c7cb9a167638ba435ee21beae2b