fix(pingcap/tidb): fix the cache too large problem

[wuhuizuo]

do not cache the third party bin in prepare stage.

Signed-off-by: wuhuizuo wuhuizuo@126.com

[ti-chi-bot[bot]]

I have already done a preliminary review for you, and I hope to help you do a better job.

Based on the pull request title and description, the key change in this pull request is to fix a cache problem by not caching the third-party bin in the prepare stage. Looking at the diff, there are a few changes made to the pull_br_integration_test.groovy file. The Prepare stage has been modified to remove the caching of the third-party bin and some other unnecessary cache. Instead, the third-party bin is downloaded and copied to the bin directory. Additionally, a new Prepare stage has been added before the Test stage.

One potential problem with this change is that the download of the third-party bin in the new Prepare stage is not cached. This could lead to longer build times if the download takes a long time. It might be better to cache the downloaded bin if possible.

To fix this potential problem, we can add a cache for the downloaded third-party bin in the new Prepare stage. We can modify the dir('third_party_download') block to cache the third_bin directory. Here is an example:

dir('third_party_download') {
    cache(path: "./third_bin", key: prow.getCacheKey('third-party-bin', REFS)) {
        retry(2) {
            sh label: "download third_party", script: """
                chmod +x ../tidb/br/tests/*.sh
                ${WORKSPACE}/tidb/br/tests/download_integration_test_binaries.sh master
                rm -rf bin/ && mkdir -p bin
                mv third_bin/* bin/
                ls -alh bin/
                ./bin/pd-server -V
                ./bin/tikv-server -V
                ./bin/tiflash --version
            """
        }
    }
}

With this change, the downloaded third-party bin will be cached and reused in subsequent builds if the same cache key is used.

[wuhuizuo]

tested in: https://do.pingcap.net/jenkins/job/pingcap/job/tidb/job/pull_br_integration_test/1368/

with just G00 enabled in matrix.

[wuhuizuo]

I have already done a preliminary review for you, and I hope to help you do a better job.

Based on the pull request title and description, the key change in this pull request is to fix a cache problem by not caching the third-party bin in the prepare stage. Looking at the diff, there are a few changes made to the pull_br_integration_test.groovy file. The Prepare stage has been modified to remove the caching of the third-party bin and some other unnecessary cache. Instead, the third-party bin is downloaded and copied to the bin directory. Additionally, a new Prepare stage has been added before the Test stage.

One potential problem with this change is that the download of the third-party bin in the new Prepare stage is not cached. This could lead to longer build times if the download takes a long time. It might be better to cache the downloaded bin if possible.

To fix this potential problem, we can add a cache for the downloaded third-party bin in the new Prepare stage. We can modify the dir('third_party_download') block to cache the third_bin directory. Here is an example:

dir('third_party_download') {
    cache(path: "./third_bin", key: prow.getCacheKey('third-party-bin', REFS)) {
        retry(2) {
            sh label: "download third_party", script: """
                chmod +x ../tidb/br/tests/*.sh
                ${WORKSPACE}/tidb/br/tests/download_integration_test_binaries.sh master
                rm -rf bin/ && mkdir -p bin
                mv third_bin/* bin/
                ls -alh bin/
                ./bin/pd-server -V
                ./bin/tikv-server -V
                ./bin/tiflash --version
            """
        }
    }
}

With this change, the downloaded third-party bin will be cached and reused in subsequent builds if the same cache key is used.

I known, but the issue is to workaround the cache size limit problem.

[ti-chi-bot[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - **[pipelines/OWNERS](https://github.com/PingCAP-QE/ci/blob/main/pipelines/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment