We kept seeing CI running external PRs over and over. Even though everything seemed to be fine. I realized that CI was deleting results every evening and then re-running them and never getting caught up.
The root of the problem was found to be an issue during cleanup. When CI runs cleanup, it pulls the current branch list, PR list, and release list. It then loops over all results files in the results repo and verifies the files match an existing branch, PR, or tag. If it doesn't, it deletes the results, keeping the dashboard and results repo updated and clean. However, there was an issue with the PR list. It by default only grabs 30 pull requests. We often have more than 30 pull requests open, so results for PRs older than 30 were being pruned because they didn't belong to any known PR.
This fix bumps the value up to 50...sheesh we shouldn't have near that much...so now the results should stay as long as the PR is open.
Some other minor cleanups along the way:
Capture Octokit::NotFound when searching for the .decent_ci-limits.yaml file, no need to throw an error
Only look up the branches, releases, and pull requests once at the beginning of the cleanup process, rather than finding them on each individual file. There shouldn't be any other cleanups going on at the same time so I think this should be fine. And hopefully it will speed up the cleanup process as a whole.
We kept seeing CI running external PRs over and over. Even though everything seemed to be fine. I realized that CI was deleting results every evening and then re-running them and never getting caught up.
The root of the problem was found to be an issue during cleanup. When CI runs cleanup, it pulls the current branch list, PR list, and release list. It then loops over all results files in the results repo and verifies the files match an existing branch, PR, or tag. If it doesn't, it deletes the results, keeping the dashboard and results repo updated and clean. However, there was an issue with the PR list. It by default only grabs 30 pull requests. We often have more than 30 pull requests open, so results for PRs older than 30 were being pruned because they didn't belong to any known PR.
This fix bumps the value up to 50...sheesh we shouldn't have near that much...so now the results should stay as long as the PR is open.
Some other minor cleanups along the way:
Octokit::NotFound
when searching for the.decent_ci-limits.yaml
file, no need to throw an error