coverallsapp / github-action

Coveralls Github Action
https://coveralls.io
MIT License
458 stars 76 forks source link

Are multiple jobs with the same flag-name supported? #143

Closed mdantonio closed 1 year ago

mdantonio commented 1 year ago

I have some parallel tests all configured with the same flag-name.

The workflow is similar to:

    strategy:
      matrix:
        test_n:
        - "test-1"
        - "test-2"
        - "test-3"
        .....
    - name: Run tests
       run: ....

    - name: Send coverage
      uses: coverallsapp/github-action@master
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        parallel: true
        flag-name: odm

The result is the following, it seems that only one job is considered (I don't know exactly if the first or the last sent)

Screenshot 2023-02-06 alle 13 13 10

(note: in this example the resulting job id is .10, in a previous run was .6 ... so I think that it corresponds to the first/last completed test.)

On the opposite I tried to send the coverage by using the python uploader and this setup is working as expected, here a screenshot for the UI. You can see the same flag name associated to multiple Job IDs.

Screenshot 2023-02-06 alle 13 15 04

What is weird is that if you look at the number of files in the first screenshot they seem to be aggregated (420 files, much more of any other number of files in the second screenshot) but the source files tree (not included in these screenshots) only shows file covered from 1 test (on the opposite the tree obtained by using the python uploader is showing all files aggregated).

So my question is: is that setup supposed to be supported by the coveralls github action? And why I'm observing such difference if compared to the python uploader?

mrexox commented 1 year ago

Hey! Job flags are supposed to be unique within one build, so they can be carry-forwarded with carryforward option. I'm not sure why there is such a difference. Might be a difference with references of build. If you could provide a repo link and the python uploader link I can dig into it.

But the main advise is to use unique job flags.

mdantonio commented 1 year ago

Thank you, If the expectation is to have unique flags then I think that this issue can be closed.