Regarding coverage...

[pboling]

Just read the blog post on the migration from http-2-next back here to http-2, and saw the request for assistance with coverage.

It may not be everyone's cup of tea, but I consolidated all of my knowledge on how I do coverage on my gems into a gem, so I could share the work with myself, and not redo it for every repo. The result is a "one four line" config that, when used with github actions like this, will give you CodeClimate Coverage and Maintenance badges, as well as CodeCov badges, and which can then integrate PR comments related to coverage changes, and it can be easily instrumented to fail the build when a coverage threshold isn't met (line or branch). All instrumentation is via ENV variables. I haven't had any major adopters except for @sunny, but I think it provides a lot of value, and to your point about community re-work, it avoids me having to resolve the same CI issues in every project.

It provides a coverage rake task, and is able to entirely configure every coverage output format (and all of them at once if you want) I am aware of, including:

      FORMATTER_PLUGINS = {
        # HTML for Humans
        html: {
          type: :html,
          klass: "HTMLFormatter",
          lib: "simplecov-html",
        },
        # XML for Jenkins
        xml: {
          type: :xml,
          klass: "CoberturaFormatter",
          lib: "simplecov-cobertura",
        },
        # RCOV for Hudson
        rcov: {
          type: :rcov,
          klass: "RcovFormatter",
          lib: "simplecov-rcov",
        },
        # LCOV for GCOV
        lcov: {
          type: :lcov,
          klass: "LcovFormatter",
          lib: "simplecov-lcov",
        },
        # JSON for CodeClimate
        json: {
          type: :json,
          klass: "JSONFormatter",
          lib: "simplecov_json_formatter",
        },
        # TTY / Console output
        tty: {
          type: :tty,
          klass: "Console",
          lib: "simplecov-console",
        },
      }

https://gitlab.com/kettle-rb/kettle-soup-cover (config class)

The ENV variables used for instrumentation:

K_SOUP_COV_COMMAND_NAME
K_SOUP_COV_DEBUG
K_SOUP_COV_DIR
K_SOUP_COV_DO
K_SOUP_COV_FILTER_DIRS
K_SOUP_COV_FORMATTERS
K_SOUP_COV_MERGE_TIMEOUT
K_SOUP_COV_MIN_BRANCH
K_SOUP_COV_MIN_HARD
K_SOUP_COV_MIN_LINE
K_SOUP_COV_MULTI_FORMATTERS
K_SOUP_COV_PREFIX
K_SOUP_COV_USE_MERGING
K_SOUP_COV_VERBOSE

I'd love to hear your thoughts, and I would LOVE if the community had a better place to amass shared understanding of "how to do code coverage in Ruby with config for both local and CI".

[HoneyryderChuck]

I, thank you for taking the time to read the post and propose a solution!

Bear in mind that, although not rocket science, the CI config is somewhat unconventional: tests are run against multiplle versions and platforms in ruby, and generate a coverage config file; a post-step collects all generated coverage files, and collates them to generate the aggregate coverage + reports.

From the current CI suite standpoint, I guess the requirements for the missing functionality are:

• publish aggregate coverage reports somewhere public
• generate a badge
• link badge in readme to public reports

I had a look at your kettle-soup-cover. I may not have fully understood the full scope of it, however I can say that this repo wouldn't necessarily benefit from supporting multiple output formats (it only requires json/html, both supported OOTB by simplecov), nor targetting multiple coverage services (I'd prefer not relying on any and just host it in github somehow, I just didn't find docs on how to do it, but I can also be convinced that it's not worth the trouble, in which case I'd be open to integrate just a single third party for it). It also seems to integrate directly in the helper.rb file somehow, and as per above, the rake task collating several coverage outputs is the one doing the heavy lifting, not sure whether this would support it either.

I'd also prefer to keep this barebones (if reasonable) and just use simplecov, if possible.

LMK if this makes sense, and whether I didn't get confused with what features your gem provides.

[pboling]

I agree, there's a significant gap between what this gem needs and what my coverage gem provides. I'd like to add more functionality around automating coverage aggregation and publishing, if there is a normalized "best practice" way we could accomplish that. Perhaps a GitHub Action could be built to do that... I'll close this for now, but love discussion of ways to consolidate community understanding of the "hard fiddly bits" of what we do.