Upload metrics from PR CI as artifacts in GitHub Actions

[jdno]

We collect a lot of metrics about each CI run. In builds run by bors within the rust-lang-ci organization, we upload these metrics to S3 and make them available at https://ci-artifacts.rust-lang.org/rustc-builds/{run.sha}/metrics-{name}.json. This provides a great tool to analyze the efficiency of our builds and continually optimize them.

For builds run on pull requests in the public rust-lang organization, we don't upload the metrics to S3 because we don't want to add a token with write permissions to S3 to the public environment.

Instead of discarding the metrics after each run, we should upload them as artifacts in GitHub Actions. This will allows us to store them somewhere without having to grant the jobs any additional permissions, and download and analyze them similar to the metrics collected by bors.

[meysam81]

Hey @jdno,

I'd like to give this a try.

I realized you mentioned "rust-lang organization public repos". Does that imply that creating a workflow file in here is enough to resolve this requirement? (and then later on, using such workflow from other repositories with uses: rust-lang/simpleinfra/github-actions/example@v1?)

[Kobzol]

Rather than a new workflow, I think that this should be added to the existing CI code. There is code at the end of src/ci/run.sh (IIRC) that uploads the metrics to S3. If we find that S3 is not available (or that we're in a PR CI run, I think that there is now a dedicated env variable that tells us that), the metrics file should instead be uploaded as a workflow artifact.

[Kobzol]

Sorry, I actually misunderstood your comment, I thought that you wanted to create a new workflow in rust-lang/rust. So yeah, it could be a workflow in simpleinfra, and then it would be added to our CI workflow as another step after the run step. We have to double check that the metrics file is not deleted in run.sh though and that it's available to following GHA steps.

[jdno]

Sorry that the issue description wasn't very specific. We only really care about the metrics from rust-lang/rust. I'm not even sure if other repositories produce similar metrics during their builds.

We already have a step that uploads documentation from the pull request builds:

- name: upload artifacts to github
  uses: actions/upload-artifact@v3
  with:
    # name is set in previous step
    name: ${{ env.DOC_ARTIFACT_NAME }}
    path: obj/artifacts/doc
    if-no-files-found: ignore
    retention-days: 5
  <<: *step

I think that we just need to add another step that uploads the files that upload-artifacts.sh would upload as well. Since that is only required for pull requests, it should have an if condition that limits it to pull request events.

And since we only need a single step, I would avoid creating a new GitHub Action for this.