lemurheavy / coveralls-public

The public issue tracker for coveralls.io
http://coveralls.io
124 stars 7 forks source link

[docs] Does not explain require `cov2lcov` (golang) #1510

Closed AlbinoGeek closed 2 years ago

AlbinoGeek commented 3 years ago

Summary

To make Coveralls accept Golang's default coverage report format, one must insert a cov2lcov step in their build, which was not explained whatsoever in the documentation on Coveralls website, nor on the README for the GitHub Coverall Action.

What is lcov?

I would love to know (at least, in a summary sense) why this specialized coverage format is required, and which languages provide it out of the box. Golang does not appear to be one of these languages, hence the additional action cov2lcov being required.

Working Example

https://github.com/AlbinoGeek/sc2-rsu/blob/gui/v2/.github/workflows/codeql-analysis.yml#L84-L95

afinetooth commented 3 years ago

Hi @AlbinoGeek,

Before I address your issue, let me see if I can save you time with a recommendation, first:

Most new users pick a community-created integration from the list of language integrations in the left-side navigation panel of our website:

Screen Shot 2020-12-17 at 4 04 41 PM

Given that, most Go users start with the integration(s) here: https://docs.coveralls.io/go

Of which, goveralls is the most popular and actively supported project.

From browsing the goveralls project, I don't see lcov anywhere, so its possible that project chose to translate from a native reporting standard directly into the JSON it POSTs to the coveralls API.

For that reason, your easiest path might be to add that integration to your project.

In the meantime, if you do want to roll your own integration:

LCOV - You're right about the requirement of an LCOV-format coverage report, but only for the tools and integration components we've contributed to the community, such as the Coveralls Github Action and our Universal Coverage Reporter.

We settled on LCOV for those tools because of its long history, having been created to complement gcov, and the fact that most languages created later wrote to that standard to make their coverage reports backwards compatible. Go may be an exception.

Basic Recipe - Considering most projects are running in CI, here's the basic recipe for an integration with Coveralls:

Basic Procedure - In the simplest possible terms, assuming you've committed changes to your project's repo and triggered a new build in CI, these are the steps your integration must complete in CI:

  1. Generate Coverage Report - Some command runs your test suite against your codebase and, by virtue of your project including a language-specific coverage library, exports a code coverage report. (LCOV is the format we look for, but other formats are OK as long as the next step completes.)
  2. Upload Coverage Report - Some script running in CI---part of your third-party integration or not---reads in the coverage report, converts it to JSON and POSTs it to the coveralls API at https://coveralls.io/api/v1/jobs.

So given the above, LCOV is only a requirement if you want to leverage one of the tools we've contributed, as you did with the Coveralls Github Action (which handles Step 2).

It sounds like you've made progress on that conversion step with cov2lcov. Here's another project along the same lines.

Your point is well-taken about our documentation lacking in places like this. While most detailed usage instructions can be found in the READMEs for individual integrations, some general documentation like this would be helpful. We're aware and are working on an update to our docs due early next year.

stale[bot] commented 2 years ago

This issue has been automatically marked for closure because it has not had recent activity. It will be closed if no further activity occurs. If your issue is still active please add a comment and we’ll review as soon as we can. Thank you for your contributions.

AlbinoGeek commented 2 years ago

We were using goveralls, that's the most confusing part.

Either way, we've stopped using coveralls since this.

afinetooth commented 2 years ago

Hi, @AlbinoGeek. I see. Well, I'm sorry to hear I wasn't able to help you get it working.

Looking closer at the README for the goveralls project, I see there's a related Github Action dedicated to integrating Go projects with Coveralls through Github Actions:

It's here: https://github.com/marketplace/actions/actions-goveralls

I think that would be your shortest path to getting your project integrated with coveralls.io and would not require the extra step of converting the standard Go coverage format to LCOV. (To clarify, the Coveralls Github Action would require that extra step because it expects LCOV, but the goveralls action does not.)