Add codecov support

[redboltz]

@ygj6 could you try codecov support? I guess that it requires some github setting. I will do this.

[redboltz]

@ygj6 , I merged #833 then codecov upload is triggered.

But I got the following error. https://github.com/msgpack/msgpack-c/runs/388300918#step:7:29297

I got Upload Token from https://codecov.io/gh/msgpack/msgpack-c and set xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx format hex value as CODECOV_TOKEN at https://github.com/msgpack/msgpack-c/settings/secrets.

I'm not sure why upload is failed. Any ideas?

[ygj6]

I do the same as you. Maybe you should try Regenerate to get a new token in https://codecov.io/gh/msgpack/msgpack-c/settings

[redboltz]

Ok, I will try to regenarate.

BTW, I added github secret CODECOV_TOKEN to my own repository then I push master. The coverage data successfully uploaded.

https://codecov.io/gh/redboltz/msgpack-c

[redboltz]

I regenerated the secret but got the same result.

I tried some updates on my pull request #834

I got the following result. In my own repository the coverage upload is successfully finished but in msgpack-c, it failed.

My own repository (redboltz)

https://github.com/redboltz/msgpack-c/commit/15674b681f821430ff2e1b825da9f27f0db5d10c/checks?check_suite_id=399227805

msgpack-c repository (msgpack)

https://github.com/msgpack/msgpack-c/commit/15674b681f821430ff2e1b825da9f27f0db5d10c/checks?check_suite_id=399227826

I tried to echo ${{ secret.CODECOV_TOKEN }}. In msgpack-c, it returns nothing (empty string).

I guess that it is permission problem.

I found the following document on github.

Secrets

Secrets are environment variables that are encrypted and only exposed to selected actions. Anyone with collaborator access to this repository can use these secrets in a workflow.

Secrets are not passed to workflows that are triggered by a pull request from a fork. Learn more.

[redboltz]

@nobu-k , do you know something about this?

[redboltz]

I tried hard coding CODECOV_TOKEN, then upload is successfully finished on msgpack-c repository:

https://github.com/msgpack/msgpack-c/commit/9a92fd449e0ba7486a403fb43032d1058289f47d/checks?check_suite_id=399251857#step:7:461

So the problem is "How to access secret on https://github.com/msgpack/msgpack-c/ from pull requests". I'm not sure when I merge the PR to master but CODECOV_TOKEN access is failed.

Anyway, I think that it is worth trying to reserach other open source project on github approach.

[redboltz]

A workaround I just come up with is using travis-ci for codecov. codecov doesn't require token.

There is the following document on codecov setting page:

Repository Upload Token

The token below is used exclusively for uploading coverage reports.

Note: Token not required for public repositories uploading from Travis, CircleCI or AppVeyor.

[redboltz]

https://github.community/t5/GitHub-Actions/Make-secrets-available-to-builds-of-forks/td-p/30678

[redboltz]

https://community.codecov.io/t/whitelist-github-action-servers-to-upload-without-a-token/491/13

[redboltz]

Here is the current conclusion.

• codecov will support tokenless upload when github actions will release API.
  • It will be supported early this year.
• Before tokenless upload is supported, msgpack-c has two options.
  1. Use travis-ci for codecov.
  2. Use hard-coded codecov token.

I'd like to use hard-coded token. I consider the risk. Anyone can see the token, so anyone can upload coverage data. It is not good but the damage is very limited I guess. If something wrong happened, I can re-generate the token to stop something wrong thing.

@ygj6, what do you think ?

[ygj6]

Coveralls is an alternatives to Codecov, I will try it. your opinion?

[redboltz]

I don't know much about Coveralls but it is worth trying.

I just glanced the page https://github.com/marketplace/actions/coveralls-github-action, GITHUB_TOKEN seems to be required.

I need filtering target functionality as #834 The result is https://codecov.io/gh/msgpack/msgpack-c/pull/834/tree

If you try it, please let me know the result.

[redboltz]

I guess that as long as the coverage service requires to provide github secret, the problem isn't solved.