Coverage report doesn't appear on the Overview tab

Finesse commented 5 years ago

I've set up a GitHub Actions workflow to report my test code coverage to Code Climate. The workflow uses this action. As a result, the coverage report appears in the project settings:

But doesn't appear on the Overview tab in Code Climate:

I've directed this issue to the Code Climate support, they said that the branch name (provided by this action) must me master instead of refs/heads/master.

The full conversation

Code Climate: > Hi Surgie, > > Thanks for reaching out! Happy to help and point you in the right direction. > > I haven't seen this GitHub Action before -- pretty cool! I'm not sure how to make changes to the GitHub Action to make this work for you, but here's what we see on our side: > > On the backend, I do see that we're successfully receiving payloads for this repo (last commit **f40b844b**); however, they each identify your **master** branch as: > > `refs/heads/master` > > To help resolve this error, we generally recommend making the path above ^^ relative to the project root. > > - You can do this by adding the `--prefix` flag to your to your CI's coverage configuration. > - More information is in our docs [here](https://docs.codeclimate.com/docs/codeship-ci-test-coverage-example#section-project-configuration). > - It should look something like `--prefix /[path to directory]` > > This removes absolute paths in coverage payloads to make them relative to the project root. This is usually the directory in which the tests were run. > > (You could also instruct your test setup to start at the root of your project.) > > Again, I'm not sure how to make the above work with the GitHub Action. You might need to open an issue with the maintainer on the project itself: https://github.com/paambaati/codeclimate-action/issues > > -> Once you've made changes to your setup, could you try pushing a brand new commit to **master**? > > Let me know if you have any questions, or if there's anything else I can help with. I'll be standing by to help further. > > Thanks, > Emily I: > Hi! > > I used the same PhpUnit setup and CLI command with a plain cc-test-reporter command in another repository and it worked fine. Also the reported coverage is shown in [the Code section](https://codeclimate.com/github/Finesse/SwiftMailerDefaultsPlugin/code) but not in [the Overview section](https://codeclimate.com/github/Finesse/SwiftMailerDefaultsPlugin). > > Will the problem be solved if my action sends the same report but with branch `master` instead of `refs/heads/master`? > > I’ll make an issue in the action repository and attach our conversation. Code Climate: > Hi Surgie, > > Thanks for the additional info. > > > Will the problem be solved if my action sends the same report but with `branch` master instead of `refs/heads/master`? > > Yes! That should do the trick 👌 > > Let me know once you're able to get that working! I'd love to make some notes for troubleshooting for anyone else who runs into the same. > > Thanks, > Emily > > PS: I couldn't find the output for the coverage info it in your [Workflow logs](https://github.com/Finesse/SwiftMailerDefaultsPlugin/actions), but I did some branch name troubleshooting earlier with another GitHub Workflow user. his setup reported branch names to be HEAD and I recommended this [stackflow conversation](https://stackoverflow.com/questions/58033366/how-to-get-current-branch-within-github-actions) > > (here's the coverage-related [Workflow logs from the other user](https://github.com/NullVoxPopuli/emberclear/commit/bef39135f90109323089dd3e6e663311bf00f01e/checks?check_suite_id=274295145#step:6:508))

Please also consider that a branch can have a slash in its name, e.g. a gitflow-style branch feature/name.

paambaati commented 5 years ago

@Finesse Thanks for the detailed report. Let me debug this; I’ll try to ship a fix this weekend.

paambaati commented 5 years ago

@Finesse Do you mind testing my fix? You can use the branch branch-name-fix-42.

In your Actions YML, simply replace uses: paambaati/codeclimate-action@v2.2.1 with uses: paambaati/codeclimate-action@branch-name-fix-42

Finesse commented 5 years ago

@paambaati The action fails: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/276345629#step:5:6

paambaati commented 5 years ago

Looks like using the branch doesn’t work. Let me merge this and you can perhaps try master.

Finesse commented 5 years ago

@paambaati I think this is not the reason. Your action.yml points to lib/main.js while this file doesn't exist in the repository.

paambaati commented 5 years ago

@Finesse lib/main.js is generated during build, so that isn't an issue.

I just pushed a new release (v2.2.2) with the fix. Can you try that and let me know if it works?

Finesse commented 5 years ago

@paambaati Still fails: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/276363513#step:1:7

paambaati commented 5 years ago

@Finesse Sorry for the back and forth! Can you try release v2.2.3?

Finesse commented 5 years ago

@paambaati Fails: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/276484319#step:1:7

paambaati commented 5 years ago

@Finesse Can you try v2.2.4?

Finesse commented 5 years ago

@paambaati The same, fails: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/276563757#step:1:7

paambaati commented 5 years ago

@Finesse I've entirely removed the dependency causing this issue. Can you try v2.2.5?

Finesse commented 5 years ago

@paambaati Fails, but the error is new: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/278796185#step:1:9

paambaati commented 5 years ago

@Finesse Sorry this is taking a lot of time, but can you now try v2.2.6?

Finesse commented 5 years ago

@paambaati Fails with another error: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/278808824#step:5:6

vinicius33 commented 5 years ago

Hey guys, I'm having the same issue. @paambaati let me know if I can help you somehow.

Thanks!

paambaati commented 5 years ago

For now, I'd recommend sticking to v2.2.1. I'm still not sure what's causing this problem, so @vinicius33, if you can try to include more debug steps to the latest master and help with a root-cause, I'd appreciate it a lot!

vinicius33 commented 5 years ago

TBH @paambaati, it seems that everything's alright now. However, for some reason, lib/main.js isn't being built when you publish the action to the store. I'd assume that something went wrong with your release workflow. Perhaps this action was the problem ?

paambaati commented 5 years ago

@vinicius33 That action actually does build lib/main.js - see https://github.com/paambaati/codeclimate-action/blob/gh-actions/lib/main.js

It pushes the actual release output to the gh-actions branch.

vinicius33 commented 5 years ago

@paambaati I see... that's weird. Maybe a new release might help?

vinicius33 commented 5 years ago

@vinicius33 That action actually does build lib/main.js - see https://github.com/paambaati/codeclimate-action/blob/gh-actions/lib/main.js

It pushes the actual release output to the gh-actions branch.

@paambaati it seems like the latest commit for this file was 2 days ago. 🤔

paambaati commented 5 years ago

@vinicius33 Only because the last release (see https://github.com/paambaati/codeclimate-action/issues/42#issuecomment-547256277) was 2 days ago.

paambaati commented 5 years ago

@Finesse @vinicius33 Can you retest v2.6.6? I just pushed out the same release again, and it seems to work.

Finesse commented 5 years ago

@paambaati It doesn't fail: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/282740248#step:5:0

But the branch name still has refs/heads/

paambaati commented 5 years ago

@Finesse Phew, at least we're able to proceed now.

Can you re-run it? I just pushed v2.2.6 again with another fix.

Finesse commented 5 years ago

@paambaati The same:

BTW, could you please add an action parameter to turn on/off the Code Climate debug? It will add additional information for Code Climate support:

To get additional output in the logs (essentially, see what's sent in the coverage payload), you can:

add the --debug flag to your CI config

For the debug option:

If you're running your tests in single builds, use after-build --debug.

For multiple or parallel tests, please use both format-coverage --debug and upload-coverage --debug.

Finesse commented 5 years ago

The regexp must be /^refs\/heads\// instead of /^refs\/head\//

paambaati commented 5 years ago

The regexp must be /^refs\/heads\// instead of /^refs\/head\//

@Finesse I just realized that! Let me do that first.

BTW, could please add an action parameter to turn on/off the Code Climate debug?

Doable, let me include that too in the next release.

paambaati commented 5 years ago

@Finesse Can you try v2.3.0 now? It should fix this issue and also include --debug support.

The README is also updated with examples.

Finesse commented 5 years ago

@paambaati The coverage has been successfully reported 🎉

But the debug option doesn't work: https://github.com/Finesse/SwiftMailerDefaultsPlugin/runs/282820415#step:5:4. Maybe you need to add it to action.yml.

paambaati commented 5 years ago

@Finesse Pushed v2.3.0 again. Can you try?

paambaati commented 5 years ago

@Finesse Looks like it worked. Can I close this issue?

Finesse commented 5 years ago

@paambaati I'm performing the final test on the master branch. I'll let you know when it finishes.

Finesse commented 5 years ago

@paambaati Everything looks good: PR auto comment, debug information and the Overview tab. The issue can be closed.

Thank you!

paambaati / codeclimate-action

Coverage report doesn't appear on the Overview tab #42