Monorepo support: Viewing coverage by flag

[EvHaus]

Hi there,

We're trialing Coveralls. We work in a monorepo so it's great that Coveralls supports the job "flag-name" values so that we can submit coverage for different packages inside our monorepo. However, the Coveralls UI doesn't seem to handle it very gracefully. Here's what the UI looks like when we have 2 different flags being used:

Looks like the chart ping-pongs between Package 1 coverage % and Package 2 coverage % resulting in this zig-zag chart which isn't very useful in terms of reviewing the coverage trends.

We have Coveralls configured with GH Actions using coveralls-github-action with a separate GH Actions workflow for each of our packages. Each workflow has something like this on the end:

- name: Coveralls
  continue-on-error: true
  uses: coverallsapp/github-action@master
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    flag-name: packageA

Since each package is managed via a separate workflow, we don't set parallel: true (and consequently never fire parallel-finished). Is that a requirement when submitting coverage for multiple flags? In our case, I'm struggling to think when we would call parallel-finished since there's no way to easily know when ALL GH actions workflows across the repository are done. They're all run independently.

Is it possible to be able to filter the UI by flag name, so that we could see how each of our monorepo packages are trending?

[afinetooth]

Hi @EvHaus.

To answer your question: Yes, flag-name is meant to be used in the context of parallel builds, for which configuration is fairly prescribed. We expect that you will be running a single workflow, using a so-called matrix build, and that each parallel job in your build (each with parallel: true) will be coming to coveralls with the same workflow ID.

In addition, we need you to close the parallel build with the parallel-finished step (which sends us the parallel build webhook), which tells us you're finished and we can close the build (and start calculating aggregate coverage, etc). And we know which build to close because one of the params in that call is the build_num, which for Github Actions is the env var, GITHUB_RUN_ID (unique to each workflow, and what I'm referring to as the "workflow ID" above).

It sounds like each of your builds is coming in as "non-parallel", with only one job per build, and that's because we recognize each workflow ID as a different build. That's why you're getting the coverage for one package in one build, and the other in the next.

If you set up your CI config to run as described above, or can figure out a way to meet the same requirements across multiple files (not that I know of one), then you will start seeing parallel builds and they'll be consistent in terms of coverage. Meaning they will only change to the extent that actual coverage changes between builds.

Is it possible to be able to filter the UI by flag name, so that we could see how each of our monorepo packages are trending?

Yes and no. Every job in a parallel build gets its own page in the UI. In other words, in your new build pages, you'll see a list of jobs in the JOBS section of the page and you'll be able to click on each one to see a page of coverage stats filtered for that job. (You can actually do this now for your non-parallel builds. Just click on the linked name of your one job to see an example of a job page.)

[jalaziz]

We're also looking at coveralls and use a monorepo.

In our case, each component is developed independently, so having one matrix workflow is not really an option (without significantly slowing down development).

Having support for each "flag" as a separate view would be nice.

[aweingarten]

We are looking for a solution here to

[AlekSi]

Same here. It would be nice to support parallel GitHub Actions workflows in addition to parallel jobs in a single workflow. Given that you know the PR number and commit hash, I think it should be possible. codecov.io supports that, for example.