This PR adds a Disk Cleanup step as the first step in the lint CI job to allow for more virtual memory space during linting and limits the concurrency of both of the first analysis cache construction and the second full linting phases to 3. Previously, the second phase was run with the max available concurrency as we had observed no issues with the standard Github hosted runners (label ubuntu-22.04) but in a test PR, we had some mixed results with this phase. So we will try configuring the second phase by limiting its concurrency. Please see this comment for some further details.
Furthermore, the first phase is now always run. In the experiments we did with this PR, even there's an analysis cache hit and the cache is restored from a previous run, the second phase can run out of memory even with a concurrency of 1. With the experiment done here, running the cache construction phase again, which employs the build constraints, on top of the restored analysis cache allows the full linting phase to complete successfully. Please note that the full linting phase does not employ the build constraints, more precisely it uses the all build constraint, which effectively lints the whole code base (except the two hot spots discussed in the previous PRs). Looks like even there's cache hit, it's imperative to run the first phase with a subset of the code base (partitioned with the build constraints) is imperative for decreasing the memory needs of the second (full linting) phase.
I have:
[x] Run make reviewable test to ensure this PR is ready for review.
How has this code been tested
A successful run (with a concurrency of 1) has been observed here and another successful re-run (again with a concurrency of 1) is here. This run uses a concurrency of 3 and decreases the linter runner time from ~45 min to 32 min at the expense of increased memory usage. We will continue with limiting the concurrency to 3 and we may revisit this if the linter starts failing due to high memory consumption.
Description of your changes
This PR adds a
Disk Cleanupstep as the first step in thelintCI job to allow for more virtual memory space during linting and limits the concurrency of both of the first analysis cache construction and the second full linting phases to 3. Previously, the second phase was run with the max available concurrency as we had observed no issues with the standard Github hosted runners (labelubuntu-22.04) but in a test PR, we had some mixed results with this phase. So we will try configuring the second phase by limiting its concurrency. Please see this comment for some further details.Furthermore, the first phase is now always run. In the experiments we did with this PR, even there's an analysis cache hit and the cache is restored from a previous run, the second phase can run out of memory even with a concurrency of 1. With the experiment done here, running the cache construction phase again, which employs the build constraints, on top of the restored analysis cache allows the full linting phase to complete successfully. Please note that the full linting phase does not employ the build constraints, more precisely it uses the
allbuild constraint, which effectively lints the whole code base (except the two hot spots discussed in the previous PRs). Looks like even there's cache hit, it's imperative to run the first phase with a subset of the code base (partitioned with the build constraints) is imperative for decreasing the memory needs of the second (full linting) phase.I have:
make reviewable testto ensure this PR is ready for review.How has this code been tested
A successful run (with a concurrency of 1) has been observed here and another successful re-run (again with a concurrency of 1) is here. This run uses a concurrency of 3 and decreases the linter runner time from ~45 min to 32 min at the expense of increased memory usage. We will continue with limiting the concurrency to 3 and we may revisit this if the linter starts failing due to high memory consumption.