pingyu commented 2 years ago

Signed-off-by: Ping Yu yuping@pingcap.com

What problem does this PR solve?

Issue Number: close #295

Problem Description: Speed up integrated tests

What is changed and how does it work?

Split integrated test cases to groups.
Add README for how to run tests (rendered).

Usage:

tests/integration_tests/run_group.sh [G00|G01| .. |G12|others]

Code changes

No code

Check List for Tests

This PR has been tested by at least one of the following methods:

Integration test
Manual test (add detailed scripts or steps below)

# tests/integration_tests/run_group.sh G00
Run cases: autorandom
=================>> Running test /cdc/tests/integration_tests/autorandom/run.sh using Sink-Type: tikv... <<=================
The 1 times to try to start tidb cluster...
start tidb cluster in /tmp/tikv_cdc_test/autorandom
...
...
[Thu Nov 17 10:46:02 UTC 2022] <<<<<< run test case autorandom success! >>>>>>
#

# make integration_test CASE="cli cli_tls"
tests/integration_tests/run.sh "cli cli_tls"
=================>> Running test /cdc/tests/integration_tests/cli/run.sh using Sink-Type: tikv... <<=================
The 1 times to try to start tidb cluster...
start tidb cluster in /tmp/tikv_cdc_test/cli
...
...
[Thu Nov 17 09:39:19 UTC 2022] <<<<<< run test case cli success! >>>>>>
=================>> Running test /cdc/tests/integration_tests/cli_tls/run.sh using Sink-Type: tikv... <<=================
The 1 times to try to start tls tidb cluster...
start tidb cluster in /tmp/tikv_cdc_test/cli_tls
...
...
[Thu Nov 17 09:41:10 UTC 2022] <<<<<< run test case cli_tls success! >>>>>>
\033[0;36m<<< Run all test success >>>\033[0m

Side effects

No side effects

Related changes

No related changes

To reviewers

Please follow these principles to check this pull requests:

Concentration. One pull request should only do one thing. No matter how small it is, the change does exactly one thing and gets it right. Don't mix other changes into it.
Tests. A pull request should be test covered, whether the tests are unit tests, integration tests, or end-to-end tests. Tests should be sufficient, correct and don't slow down the CI pipeline largely.
Functionality. The pull request should implement what the author intends to do and fit well in the existing code base, resolve a real problem for TiDB/TiKV users. To get the author's intention and the pull request design, you could follow the discussions in the corresponding GitHub issue or internal.tidb.io topic.
Style. Code in the pull request should follow common programming style. (For Go, we have style checkers in CI). However, sometimes the existing code is inconsistent with the style guide, you should maintain consistency with the existing code or file a new issue to fix the existing code style first.
Documentation. If a pull request changes how users build, test, interact with, or release code, you must check whether it also updates the related documentation such as READMEs and any generated reference docs. Similarly, if a pull request deletes or deprecates code, you must check whether or not the corresponding documentation should also be deleted.
Performance. If you find the pull request may affect performance, you could ask the author to provide a benchmark result.

(The above text mainly refers to TiDB Development Guide. It's also highly recommended to read about Writing code review comments)

pingyu commented 2 years ago

@zeminzhou @haojinming PTAL, thanks~

codecov[bot] commented 2 years ago

Codecov Report

Merging #314 (922f9bc) into main (0077830) will decrease coverage by 0.0741%. The diff coverage is 0.0000%.

Additional details and impacted files

[![Impacted file tree graph](https://codecov.io/gh/tikv/migration/pull/314/graphs/tree.svg?width=650&height=150&src=pr&token=7nmbrqKeWs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv)](https://codecov.io/gh/tikv/migration/pull/314?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv) ```diff @@ Coverage Diff @@ ## main #314 +/- ## ================================================ - Coverage 61.0618% 60.9877% -0.0742% ================================================ Files 240 240 Lines 20286 20350 +64 ================================================ + Hits 12387 12411 +24 - Misses 6779 6816 +37 - Partials 1120 1123 +3 ``` | Flag | Coverage Δ | | *Carryforward flag | |---|---|---|---| | br | `60.5320% <ø> (ø)` | | Carriedforward from [97fb51b](https://codecov.io/gh/tikv/migration/commit/97fb51be0eb35b603d29d384bbe85cd09bb0f104?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv) | | cdc | `61.1981% <0.0000%> (-0.1095%)` | :arrow_down: | | *This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv) to find out more. | [Impacted Files](https://codecov.io/gh/tikv/migration/pull/314?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv) | Coverage Δ | | |---|---|---| | [cdc/cdc/sorter/unified/backend\_pool.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9zb3J0ZXIvdW5pZmllZC9iYWNrZW5kX3Bvb2wuZ28=) | `66.9811% <ø> (ø)` | | | [cdc/cdc/sorter/unified/file\_backend.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9zb3J0ZXIvdW5pZmllZC9maWxlX2JhY2tlbmQuZ28=) | `33.3333% <0.0000%> (-0.4387%)` | :arrow_down: | | [cdc/cdc/sorter/unified/heap\_sorter.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9zb3J0ZXIvdW5pZmllZC9oZWFwX3NvcnRlci5nbw==) | `85.7142% <0.0000%> (-1.5307%)` | :arrow_down: | | [cdc/cdc/server.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9zZXJ2ZXIuZ28=) | `35.1851% <0.0000%> (-0.8338%)` | :arrow_down: | | [cdc/cdc/http\_router.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9odHRwX3JvdXRlci5nbw==) | `81.4814% <0.0000%> (-0.5186%)` | :arrow_down: | | [cdc/cdc/sorter/unified/merger.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9zb3J0ZXIvdW5pZmllZC9tZXJnZXIuZ28=) | `68.2847% <0.0000%> (-0.3237%)` | :arrow_down: | | [cdc/cdc/kv/client.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9rdi9jbGllbnQuZ28=) | `85.7348% <0.0000%> (-0.2882%)` | :arrow_down: | | [cdc/cmd/cdc/main.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NtZC9jZGMvbWFpbi5nbw==) | `100.0000% <0.0000%> (ø)` | | | [cdc/cdc/capture/capture.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9jYXB0dXJlL2NhcHR1cmUuZ28=) | `0.0000% <0.0000%> (ø)` | | | [cdc/cdc/kv/region\_worker.go](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv#diff-Y2RjL2NkYy9rdi9yZWdpb25fd29ya2VyLmdv) | `79.5833% <0.0000%> (ø)` | | | ... and [2 more](https://codecov.io/gh/tikv/migration/pull/314/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tikv) | |

purelind commented 2 years ago

/run-integration-tests