Possible to use this with Github Workflow?

[AdrianMF]

I am beginning to migrate my CI to a Github workflow. Is it possible to use this? I haven't been able to find any documentation.

[textbook]

Do you mean GitHub Actions? Have you tried https://github.com/marketplace/actions/code-climate-coverage-action?

[guilhermearaujo]

I too am having trouble getting CodeClimate to report the coverage status for a pull request.

The action @textbook mentioned uses this code to provide the env vars that cc-test-report requires:

function prepareEnv() {
  const env = process.env as { [key: string]: string };

  if (process.env.GITHUB_SHA !== undefined)
    env.GIT_COMMIT_SHA = process.env.GITHUB_SHA;
  if (process.env.GITHUB_REF !== undefined)
    env.GIT_BRANCH = process.env.GITHUB_REF;

  return env;
}

However, in my tests GITHUB_SHA and GITHUB_REF do not contain the expected values. The SHA hash is not the same as the last commit, and the REF is not the branch name, but it equals refs/pull/<pr id>/merge instead.

Without these values (or at least one of them), the pull request checks will stay eternally stuck at:

codeclimate/diff-coverage Expected — Waiting for status to be reported codeclimate/total-coverage Expected — Waiting for status to be reported

This is an example of a workflow for a Rails project:

on: [pull_request]

jobs:
  unit:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1

      - name: Set up Ruby
        uses: actions/setup-ruby@v1
        with:
          ruby-version: 2.6

      - name: Install bundler
        run: gem install bundler -v '<2'

      - name: Install dependencies
        run: bundle install

      - name: Prepare for test
        run: |
          curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
          chmod +x ./cc-test-reporter
          GIT_BRANCH=$GITHUB_REF GIT_COMMIT_SHA=$GITHUB_SHA ./cc-test-reporter before-build

      - name: Unit test with Rspec test suite
        # Coverage will be generated while running the specs using SimpleCov
        run: bundle exec rspec --format progress --profile

      - name: Report coverage
        run: GIT_BRANCH=$GITHUB_REF GIT_COMMIT_SHA=$GITHUB_SHA ./cc-test-reporter after-build -t simplecov --exit-code $?
        env:
          CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}

Update with new findings:

1. There is a variable that contains the branch name, and it is called GITHUB_HEAD_REF.

2. When the action is triggered by a pull_request event, GitHub will squash and merge this branch with the base branch, generating a new commit, thus a new hash. The commit we want (the HEAD in the branch we're merging) can be fetched by running git rev-parse origin/$GITHUB_HEAD_REF.

This is very ugly and far from being a final solution, but right now, it works:

run: GIT_BRANCH=$GITHUB_HEAD_REF GIT_COMMIT_SHA=$(git rev-parse origin/$GITHUB_HEAD_REF) ./cc-test-reporter after-build -t simplecov --exit-code $?

[robwold]

@guilhermearaujo I tried to use your solution and got this error:

fatal: ambiguous argument 'origin/': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Error: response from https://api.codeclimate.com/v1/test_reports.
HTTP 422: is invalid

Reading your post again more carefully I noticed the stipulation about the event being a pull request, and this was triggered by a commit. Could that cause this issue?

[guilhermearaujo]

@robwold I didn't stop to analyse it in depth, but I'd guess it is. When you only push a commit, and evaluate git rev-parse origin/$GITHUB_HEAD_REF, I'd say that GITHUB_HEAD_REF is blank, therefore can't rev-parse origin/<nothing>

[deivid-rodriguez]

Hei! So I had the same issues mentioned here, and found the workaround given a good starting point to fix my cases. Let me share my findings in case it helps others.

The GIT_BRANCH=$GITHUB_HEAD_REF GIT_COMMIT_SHA=$(git rev-parse origin/$GITHUB_HEAD_REF) ./cc-test-reporter workaround is good, but has some gotchas:

• The GITHUB_HEAD_REF ref needs to be fetched, otherwise you'll get the error mentioned by @robwold. If you're using actions/checkout@v1 to checkout your code, this is fetched by default, so no problem. If you're using actions/checkout@v2 or higher, this is a problem though. The workaround is to do something like git fetch --no-tags --prune --depth=1 origin +refs/heads/$GITHUB_HEAD_REF:refs/remotes/origin/$GITHUB_HEAD_REF right before. See https://github.com/actions/checkout/issues/93, which suggests this workaround.

• The upload-coverage and sum-coverage commands, don't use git information from the environment, but pick it up from the existing reports. So if you're using a format-coverage -> sum-coverage -> upload-coverage workflow, make sure you pass the proper git variables to the format-coverage command.

[deivid-rodriguez]

Here is the working workflow: https://github.com/deivid-rodriguez/pry-byebug/blob/377e5b7d229a157bb896f21d776f71fc389a5c00/.github/workflows/ubuntu.yml

[deivid-rodriguez]

Unfortunately, it doesn't seem to be working for PRs coming from external forks :(

[awentzel]

It feels as if GitHub variables should not be required to use cc-test-reporter. If they exist great, but, how do I locally test my scripts (executing on a mono-repository) when this is tightly coupled to GitHub / Actions / etc?

[Cielquan]

I tested a bit around a found a way to get the COMMIT_SHA for internal PRs and fork PR.

For a PR Github creates a merge commit (on/with a detached HEAD) and the commit message (cmt-msg) is the key. The cmt-msg is in the format: Merge <sha of latest commit on branch to merge> into <sha of latest commit of branch to merge into>. So you just have to get the cmt-msg and extract the SHA. I created a job to do this for me so I could grab the ENVVAR from this job#s outputs:

env:
  default_python_version: 3.8

jobs:
  set-git-env-vars:
    runs-on: ubuntu-latest
    outputs:
      GIT_COMMIT_SHA: ${{ steps.set-GIT_COMMIT_SHA.outputs.GIT_COMMIT_SHA }}
      GIT_BRANCH: ${{ steps.set-GIT_BRANCH.outputs.GIT_BRANCH }}
    steps:
      - uses: actions/checkout@v2

      - uses: actions/setup-python@v2
        with:
          python-version: ${{ env.default_python_version }}

      - name: Get merge commit message
        id: commit-msg
        shell: bash
        run: echo "::set-output name=COMMIT_MSG::$(git log --oneline -n 1 --format='%s')"
        if: github.event_name == 'pull_request'

      - name: Extract commit sha from merge commit msg
        id: commit-sha
        shell: python
        run: |
          import re
          import sys
          sha = re.fullmatch(
              r"^Merge ([a-z0-9]{40}) into [a-z0-9]{40}$",
              "${{ steps.commit-msg.outputs.COMMIT_MSG }}"
          )
          if sha:
              print(f"::set-output name=COMMIT_SHA::{sha.group(1)}")
          else:
              sys.exit(1)
        if: github.event_name == 'pull_request'

      - name: Set GIT_COMMIT_SHA
        id: set-GIT_COMMIT_SHA
        shell: python
        run: |
          if "${{ github.event_name }}" == "pull_request":
              sha = "${{ steps.commit-sha.outputs.COMMIT_SHA }}"
              print(f"::set-output name=GIT_COMMIT_SHA::{sha}")
          else:
              print("::set-output name=GIT_COMMIT_SHA::${{ github.sha }}")

      - name: Set GIT_BRANCH
        id: set-GIT_BRANCH
        shell: python
        run: |
          branch = "${{ github.ref }}".replace("refs/heads/", "")
          if "${{ github.event_name }}" == "pull_request":
              head_ref = "${{ github.head_ref }}"
              if head_ref:
                  branch = head_ref
          print(f"::set-output name=GIT_BRANCH::{branch}")

What I did not test yet and don't know is if the branch name without reference to the fork repository is enough for code-climate.

EDIT: fixed missing f for f-string (python) EDIT2: fixed python code

[deivid-rodriguez]

I just migrated activeadmin to use https://github.com/paambaati/codeclimate-action, and so far so good. I also tested a PR from an external fork just in case.

In case someone has a good reason to not use the action, you might want to copy the logic in there, since it seems like the correct one.

In our case, I thought we couldn't use the action because we were formatting coverage and summing+uploading it in different jobs, but it turns out that wasn't actually necessary and we moved everything to the same place, and the action worked just fine.

[tomato42]

@deivid-rodriguez setting CC_TEST_REPORTER_ID in yml file is not safe, you should use github secrets for that

[deivid-rodriguez]

It's just an ID, apparently: https://docs.codeclimate.com/docs/finding-your-test-coverage-token#should-i-keep-my-test-reporter-id-secret.

[deivid-rodriguez]

Looks like a credential, but it's not.

[deivid-rodriguez]

I guess you could try to trip up my coverage stats by submitting reports to codeclimate on behalf on my repo. So far nobody did that because the reward seems minimal :rofl:.

[sonalkr132]

you can upload the coverage folder using actions/upload-artifact@v2 and use a workflow_run (which has access to github secrets) to upload report to codeclimate. this will work with forks and keep your CC_TEST_REPORTER_ID private. check abtion/abt for implementation. PR with test coverage status.

Note: Workflow run will only be triggered if the workflow file is already on the default branch (read more)