*: caller can use a fixed TS for fast reorg

lance6716 commented 2 weeks ago

What problem does this PR solve?

Issue Number: ref #49233 close #46986

Problem Summary:

What changed and how does it work?

Check List

Tests

[x] Unit test
[ ] Integration test
[x] Manual test (add detailed scripts or steps below)
[ ] No need to test
- [ ] I checked and no code files have been changed.

GOCACHE=/tmp/failpoint-cache go build -o bin/tidb-server-failpoint -toolexec failpoint-toolexec ./cmd/tidb-server

tiup playground nightly --tiflash 0 --without-monitor --db 0

rm /tmp/tidb-{0,1}.log

GO_FAILPOINTS="github.com/pingcap/tidb/pkg/ddl/ingest/resignAfterFlush=return();github.com/pingcap/tidb/pkg/ddl/ingest/checkpointLoopExit=return();github.com/pingcap/tidb/pkg/ddl/ingest/forceSyncFlagForTest=return();github.com/pingcap/tidb/pkg/ddl/ownerResignAfterDispatchLoopCheck=return()" ./bin/tidb-server-failpoint -P 4000 --store=tikv --host=127.0.0.1 --status=10080 --path=127.0.0.1:2379 --log-file=/tmp/tidb-0.log

mysql --comments --host 127.0.0.1 --port 4000 -u root -e "set @@global.tidb_enable_dist_task = 0;set @@global.tidb_ddl_reorg_worker_cnt = 1;create table test.t(c int primary key, c2 int);split table test.t between (0) and (10000) regions 10;insert into test.t values (1,1), (3000, 3000), (6000, 6000);"

./bin/tidb-server-failpoint -P 4001 --store=tikv --host=127.0.0.1 --status=10081 --path=127.0.0.1:2379 --log-file=/tmp/tidb-1.log

mysql --comments --host 127.0.0.1 --port 4001 -u root -e "alter table test.t add unique index idx(c2);"

before

after

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

tiprow[bot] commented 2 weeks ago

Hi @lance6716. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 57.40741% with 69 lines in your changes are missing coverage. Please review.

Project coverage is 73.8742%. Comparing base (49f09fe) to head (866b8ab).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #52993 +/- ## ================================================ + Coverage 72.4413% 73.8742% +1.4328% ================================================ Files 1492 1533 +41 Lines 429013 444320 +15307 ================================================ + Hits 310783 328238 +17455 + Misses 98955 95419 -3536 - Partials 19275 20663 +1388 ``` | [Flag](https://app.codecov.io/gh/pingcap/tidb/pull/52993/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | | |---|---|---| | [integration](https://app.codecov.io/gh/pingcap/tidb/pull/52993/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `49.3033% <52.4691%> (?)` | | | [unit](https://app.codecov.io/gh/pingcap/tidb/pull/52993/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `71.2363% <34.8101%> (-0.0962%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap#carryforward-flags-in-the-pull-request-comment) to find out more. | [Components](https://app.codecov.io/gh/pingcap/tidb/pull/52993/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | | |---|---|---| | [dumpling](https://app.codecov.io/gh/pingcap/tidb/pull/52993/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `53.9957% <ø> (ø)` | | | [parser](https://app.codecov.io/gh/pingcap/tidb/pull/52993/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `∅ <ø> (∅)` | | | [br](https://app.codecov.io/gh/pingcap/tidb/pull/52993/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `46.6467% <ø> (+5.2232%)` | :arrow_up: |

lance6716 commented 1 week ago

/retest

tiprow[bot] commented 1 week ago

@lance6716: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to [this](https://github.com/pingcap/tidb/pull/52993#issuecomment-2099696003): >/retest Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

ti-chi-bot[bot] commented 1 week ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, tangenta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/pingcap/tidb/blob/master/OWNERS)~~ [D3Hunter,tangenta] - ~~[pkg/ddl/OWNERS](https://github.com/pingcap/tidb/blob/master/pkg/ddl/OWNERS)~~ [tangenta] - ~~[pkg/disttask/OWNERS](https://github.com/pingcap/tidb/blob/master/pkg/disttask/OWNERS)~~ [D3Hunter,tangenta] - ~~[pkg/lightning/OWNERS](https://github.com/pingcap/tidb/blob/master/pkg/lightning/OWNERS)~~ [D3Hunter] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

ti-chi-bot[bot] commented 1 week ago

[LGTM Timeline notifier]

Timeline:

2024-05-10 08:13:37.757198525 +0000 UTC m=+1208971.514334083: :ballot_box_with_check: agreed by tangenta.
2024-05-10 09:57:20.835816503 +0000 UTC m=+1215194.592952077: :ballot_box_with_check: agreed by D3Hunter.

lance6716 commented 1 week ago

/hold

lance6716 commented 1 week ago

/unhold

lance6716 commented 1 week ago

/cherry-pick release-8.1

ti-chi-bot commented 1 week ago

@lance6716: new pull request created to branch release-8.1: #53188.

In response to [this](https://github.com/pingcap/tidb/pull/52993#issuecomment-2105459314): >/cherry-pick release-8.1 Instructions for interacting with me using PR comments are available [here](https://prow.tidb.net/command-help). If you have questions or suggestions related to my behavior, please file an issue against the [ti-community-infra/tichi](https://github.com/ti-community-infra/tichi/issues/new?title=Prow%20issue:) repository.

pingcap / tidb