Feedback about the new pull request comment

[kylemann]

Thanks for dropping by! 👋

We've been iterating and updating the layout, summary, and copy of the pull request comment.

• What do you think about the changes?
• What would you'd like to see improved?
• What are challenges you've experienced with the report?
• How do you use the comment today?
• How could the report better help your workflow?
• Any general thoughts you'd like to share!

We greatly appreciate your time and thoughts - looking forward to hearing from you ❤

Codecov team

This issue is intended to share and collect feedback about the tool. If you have support needs or questions, please see our support page.

[gmlewis]

In the go-github repo, we consistently see messages like:

Report is 46 commits behind head on master.

without any explanation as to why this is (or how to fix it). As a result, the codecov report is not useful.

[superlopuh]

Some time in the last few weeks, the reporting of coverage by lines with diffs stopped working, as in this PR: https://github.com/xdslproject/xdsl/pull/2540 Clearly not all lines are covered by tests, and the additional info shows this, but the message disagrees. I am also not able to see the diff when clicking through to codecov.

[mortonjt]

Hi, thank you for providing this tool. We have been using this for scikit-bio

However, we have been recently having issue with codecode. Some files are mysteriously reporting little coverage, and it is not clear to us, since we have tried to create unittests that cover all of the tests (see screenshot and pull request in question here)

The only culprit that I can think of is a misconfiguration in the .codecov.yml or .coveragerc files.

Feedback would be greatly appreciated. If there is a more appropriate forum to follow up, we'll be happy to move this discussion there

[gward35]

Is there a way to customize the comment output so that we see top level coverage file coverage for branches/functions etc and not just total coverage? e.g.

Codecov Report

// all the typical stuff
...
Statements: 71.86%
Branch: 65.31%
Functions: 71.19%
Lines: 71.86%

I dont mind the patch showing the diff but at the end I also just want a plain output of the coverage in these respective buckets

[caendesilva]

The only reason I have Codecov is to see if a PR will add or remove coverage, as such I only care about that metric. Please default to have this open window open, or at least display the red/green diff in the main body! It used to be a lot better.

[codecovdesign]

thank you @caendesilva! i opened this issue to capture the feedback: https://github.com/codecov/engineering-team/issues/2022. any thoughts on the update? it adds the project change to the top level summary for quick at-a-glance review.

[caendesilva]

thank you @caendesilva! i opened this issue to capture the feedback: codecov/engineering-team#2022. any thoughts on the update? it adds the project change to the top level summary for quick at-a-glance review.

Thanks! Left a comment there.

[drazisil]

@codecovdesign I feel like zero uploaded HEAD reports should be showing as a missing head report, wdyt? :D

https://github.com/rustymotors/rusty/pull/32#issuecomment-2252489892

[karfau]

Here is another case where the report seems to calculate something strange: https://github.com/xmldom/xmldom/pull/698#issuecomment-2254651688

Not sure how it is calculated, but when 6 lines and one branch are being removed, they can of course also no longer be hit?

Or am I misunderstanding something?

I even added/extended tests in that commit.

[drazisil-codecov]

@karfau Not trying to discount your feedback. Is https://docs.codecov.com/docs/removed-code-behavior of help in your case?

[karfau]

@drazisil-codecov if I understand this documentation correctly, the default is adjust_base which shouldn't fail the check, right? And it looks as if I have not configured it: https://github.com/xmldom/xmldom/blob/master/codecov.yml

[drazisil-codecov]

@karfau I agree with you, after taking a look. @rohan-at-sentry is this a bug, or some part of the system that needs better docs?

https://app.codecov.io/gh/xmldom/xmldom/pull/698?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=xmldom is the only changed file, and it shouldn't fail project, given it's 100% covered.

[rohan-at-sentry]

@karfau - we'll look into why adjust_base isn't quite working well in your scenario. In the meantime, you can set the removed code behavior to allow a fully covered patch that removes lines to be accepted. You can edit your codecov.yml with the following setting

coverage:
  status:
    project:
      default:
        target: auto
        removed_code_behavior: fully_covered_patch 

[drazisil-codecov]

Created https://github.com/codecov/feedback/issues/463 to investigate

[moi90]

I get comments by codecov bot on PRs that were already merged a long time ago, like this one: https://github.com/morphocut/morphocut/pull/105#issuecomment-2316571448

[rohan-at-sentry]

@moi90 thanks for raising. It seems like it's related to some recent improvements we made to the notification service. We're tracking this here (https://github.com/codecov/feedback/issues/497) if you want to follow along.

[tiltowait]

codecov is great, but the report is always telling me to install the app, even though I'm pretty sure it is? Do I need to install it on all repos to suppress this message? I only need it on the one for now.

Report screenshot:

Github settings screenshot: