codeclimate / test-reporter

Code Climate Test Reporter
MIT License
154 stars 76 forks source link

Incorrect visualization of code coverage report for Golang #479

Closed georgethebeatle closed 2 years ago

georgethebeatle commented 2 years ago

Hello there,

My team has set up code coverage reporting for our project. It is a golang project and we are generating cover.out files in the gocov format. We noticed some weird looking files in our report

I am also attaching a screenshot of the relevant section in case the report is changed by future builds:

Screenshot 2021-12-01 at 17 06 32

There are a couple of things to be noted there:

We took a look at our raw cover.out file, which is in gocov format and it looks fine to us. Here is a screenshot of the same section of the code as displayed by the html report generated by go tool cover -html=cover.out:

Screenshot 2021-12-02 at 15 00 05

As you can see none of the above mentioned anomalies are present, meaning that the cover.out file is generated correctly.

We went a bit further and converted the raw file to the codeclimate format using the latest test reporter. Here is the relevant section of the json file:

    {
      "blob_id": "89e3dca239b3cb84a3a439033582a5c5782dcddd",
      "coverage": "[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,3,null,null,null,null,null,null,3,3,3,3,3,null,null,1,1,1,null,null,null,3,3,0,0,0,null,null,3,3,null,null,3,3,3,null,null,1,null,null,3]",
      "covered_percent": 85.71428571428571,
      "covered_strength": 2.1904761904761907,
      "line_counts": {
        "missed": 3,
        "covered": 18,
        "total": 21
      },
      "name": "api/apis/job_handler.go"
    },

We interpreted the coverage key of that json a a bitmap, each element corresponding to a line, where null is not a statement and a number means that the line is a statement and represents how many times the statement is visited. This interpretation aligned perfectly with the first screenshot above and confirmed our suspicion that the test reporter is not formatting the cover.out correctly.

Here is the full cover.out file for more detailed information: cover.out.txt

georgethebeatle commented 2 years ago

We opened a PR that fixes this issue

fede-moya commented 2 years ago

@georgethebeatle will be looking at this next week 👍🏼

georgethebeatle commented 2 years ago

This issue has been addressed by PR #480, closing.