search

containerbuildsystem / cachi2

GNU General Public License v3.0
7 stars 25 forks source link

Revisit publishing coverage stats online #498

Open slimreaper35 opened 5 months ago

slimreaper35 commented 5 months ago

Currently, we use the coveralls package to publish coverage stats online. It provides integration with pytest coverage. There has been no activity within the project on GitHub for a couple of years. It might be time to decide if we want to keep publishing coverage stats and, if so, look for another solution instead of coveralls.

Useful links: https://github.com/TheKevJames/coveralls-python https://github.com/TheKevJames/coveralls-python/issues/393 https://github.com/containerbuildsystem/cachi2/pull/481#discussion_r1519822031

ben-alkov commented 5 months ago

My take on this is that an HTML artifact downloadable from a GH Action is completely acceptable. IDC so much about having some website anyone can go to for coverage reporting, it's more that I don't want to have to go through the hassle of inspecting a wall of log gibberish with a microscope and scalpel...

eskultety commented 5 months ago

Haven't explored the possibilities myself yet, but if pytest doesn't support these artifacts natively, some common plugin needs to exist. Overall, @ben-alkov I like the idea of downloadable artifacts (if github allows it the same way as gitlab), so no objections from me there.

ben-alkov commented 5 months ago

if github allows it the same way as gitlab

More-or-less, IIRC. It's not rocket science :grin:

slimreaper35 commented 4 months ago

A possible 1-to-1 replacement would be Codecov

It is used for example in appstudio here

ben-alkov commented 4 months ago

Codecov looks pretty good

ben-alkov commented 4 months ago

Posting here from Slack for transparency:

Thinking out loud here, but... it should be possible to push the coverage artifacts (and unit/integration test artifacts, for that matter) to e.g. 'cachi2.github.io'... in fact, the docs specifically mention using Actions as a publishing source.

Doing so would mean we wouldn't even need Codecov.

By "coverage artifacts", I mean the outputs of the coverage html command ("unit/integration test artifacts" would require adding HTML output to pytest - out of scope for this issue); "the docs" refers to the GitHub Pages docs.

slimreaper35 commented 4 months ago

Does anyone prefer using GitHub Pages or Codecov?

⚠️ Official support for building Pages with Actions is in public beta at the moment.