post-process: support run table analyze after all tables are finished

pingcap / tidb-lightning

This repository has been moved to https://github.com/pingcap/br

Apache License 2.0

142 stars 66 forks source link

post-process: support run table analyze after all tables are finished #509

Closed glorv closed 3 years ago

glorv commented 3 years ago

What problem does this PR solve?

Add a new config analyze-at-last to allow run all table analyze tasks after all tables are finished. This is useful if TiKV instance's cpu is not rich (e.g. in standard dbaas env, TiKV is deployed with 8core cpu), thus the analyze operation will have significant impacts on SST ingest opeartions.

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Related changes

Need to update the documentation

kennytm commented 3 years ago

how about #358 🤔

glorv commented 3 years ago

how about #358 🤔

Even based on #358, we still need to do extra work to allow delay analyze phase until all other tasks are finished. Since in our current design, post-process are always executed together.

BTW, #358 has broken some integration tasks that are hard to fix, I have tried another way to pipeline the restore work based on current logic. See: https://github.com/pingcap/tidb-lightning/commit/30fe2797fdddc9eeb01203e17ec9836053492ac9 (not fully finished)

kennytm commented 3 years ago

Okay.

ti-srebot commented 3 years ago

@Little-Wallace, Thanks for your review. The bot only counts LGTMs from Reviewers and higher roles, but you're still welcome to leave your comments. See the corresponding SIG page for more information. Related SIG: migrate(slack).