search

dependabot / dependabot-core

🤖 Dependabot's core logic for creating update PRs.
https://docs.github.com/en/code-security/dependabot
MIT License
4.53k stars 944 forks source link

Measure test coverage #9203

Open JamieMagee opened 4 months ago

JamieMagee commented 4 months ago

Code improvement description

Test coverage is a simple measure of how much of the codebase has been tested. I think that setting specific coverage levels is not very helpful beyond a certain point, but it can help identify parts of the codebase that are poorly tested and may require more attention.

The most popular library for measuring code coverage in Ruby is simplecov.

yeikel commented 4 months ago

Can you assign this to me?

JamieMagee commented 4 months ago

@yeikel done. Thank you!

smoookeeey commented 4 months ago

Can you assign this to me?

JamieMagee commented 4 months ago

@smoookeeey I assigned you too, but @yeikel asked to be assigned first. Please make sure you don't duplicate any work @yeikel might have already done.

smoookeeey commented 4 months ago

Yes I will make sure not too do that

yeikel commented 4 months ago

I haven't started yet, and I won't have capacity until sometime next week. @smoookeeey, if you pick it up before that, feel free to create the PR and link it here

smoookeeey commented 4 months ago

I will look into it brother

abdulapopoola commented 4 months ago

Thanks a lot @yeikel and @smoookeeey !

JamieMagee commented 3 months ago

I just discovered that test coverage was previously added in #2576 to prevent a regression. Originally line coverage was set to 80% and branch coverage to 70%, with refuse_coverage_drop enabled to prevent any drops from occuring.

It was later gated behind the COVERAGE environment variable in #3339. However, because that environment variable wasn't set in GitHub Actions, code coverage was allowed to deteriorate. Subsequently, test coverage was removed in #7473.

yeikel commented 3 months ago

I think that rather than enforcing a certain %, we can start reporting it for awareness

JamieMagee commented 3 months ago

I agree. Reporting-only is a good start. Then something like refuse_coverage_drop to prevent backslides and act like a ratchet

yeikel commented 3 months ago

@smoookeeey Are you still working on this? Feel free to create a draft PR whenever you're ready

JamieMagee commented 2 months ago

@yeikel I think you can go ahead with this if you want.

JamieMagee commented 2 months ago

@yeikel It didn't look like you were working on this, so I created #9595