Coveralls does not work anymore

[estebanzimanyi]

Both on the main repository https://coveralls.io/builds/41566953 and in my personal repository (fork from the main repository) https://coveralls.io/builds/41566888 Previously it worked https://coveralls.io/builds/41507169 Could you please help me sorting out this issue ? Many thanks Esteban

[afinetooth]

@estebanzimanyi so you're saying that somewhere between Build 2840 and Build 2851, coveralls started reporting 0% coverage for your project(s)?

Specifically, it looks like Build 2841 is where coverage dropped from ~96% to 0%.

Do you know what changed with that PR commit? Which affected 230 files?: https://github.com/MobilityDB/MobilityDB/pull/90/files

The evidence is probably there.

But at this point, the best thing to do is examine the JSON payload you're posting to the Coveralls API.

I see your project is in C. Can you please share the integration you're using to post coverage results to coveralls? Then we can determine if that integration has a verbose mode we can engage to see exactly what you're sending for your coverage report in the JSON body of your POST.

[estebanzimanyi]

Dear James

As you rightly pointed out, there are many changes since we are organizing the files and the directories in anticipation of a future release.

Therefore, instead of trying to analyze where things are blocked, maybe the easiest way would be to "reinitialize" the repository with the new structure. Is there any document/resource that we can read to enable this kind of operation?

Regards

Esteban

---

Prof. Esteban Zimanyi Department of Computer & Decision Engineering (CoDE) CP 165/15 Universite Libre de Bruxelles Avenue F. D. Roosevelt 50 B-1050 Brussels, Belgium fax: + 32.2.650.47.13 tel: + 32.2.650.31.85 e-mail: @.*** Internet: http://cs.ulb.ac.be/members/esteban/

On Thu, Jul 22, 2021 at 7:48 PM James Kessler @.***> wrote:

@estebanzimanyi https://github.com/estebanzimanyi so you're saying that somewhere between Build 2840 https://coveralls.io/builds/41507169 and Build 2851 https://coveralls.io/builds/41566953, coveralls started reporting 0% coverage for your project(s)?

Specifically, it looks like Build 2841 https://coveralls.io/builds/41525178 is where coverage dropped from ~96% to 0%.

Do you know what changed with that PR commit? Which affected 230 files?: https://github.com/MobilityDB/MobilityDB/pull/90/files

The evidence is probably there.

But at this point, the best thing to do is examine the JSON payload you're posting to the Coveralls API.

I see your project is in C. Can you please share the integration you're using to post coverage results to coveralls? Then we can determine if that integration has a verbose mode we can engage to see exactly what you're sending for that coverage in the JSON body of your POST.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lemurheavy/coveralls-public/issues/1567#issuecomment-885098058, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGNHMBVZDTJ44ZTTCHKZFJ3TZBKVHANCNFSM5AZS2C7A .

[estebanzimanyi]

Comparying the output of the last coverage that worked

https://travis-ci.com/github/MobilityDB/MobilityDB/jobs/525690795

$ [ "$COVERAGE" == "1" ] && coveralls --exclude src/debug --exclude point/src/debug --exclude include --exclude point/include --exclude src/sql --exclude point/src/sql --exclude doc --exclude test --exclude point/test --gcov-options '-lp' -r .. -b . File '/home/travis/build/MobilityDB/MobilityDB/src/tnumber_spgist.c' Lines executed:87.12% of 264 Creating '#home#travis#build#MobilityDB#MobilityDB#build#CMakeFiles#MobilityDB.dir#src#tnumber_spgist.c.o###home#travis#build#MobilityDB#MobilityDB#src#tnumber_spgist.c.gcov' File '/usr/include/postgresql/13/server/utils/float.h' Lines executed:100.00% of 2 Creating '#home#travis#build#MobilityDB#MobilityDB#build#CMakeFiles#MobilityDB.dir#src#tnumber_spgist.c.o###usr#include#postgresql#13#server#utils#float.h.gcov' File '/usr/include/postgresql/13/server/utils/palloc.h' Lines executed:100.00% of 4 Creating '#home#travis#build#MobilityDB#MobilityDB#build#CMakeFiles#MobilityDB.dir#src#tnumber_spgist.c.o###usr#include#postgresql#13#server#utils#palloc.h.gcov' File '/home/travis/build/MobilityDB/MobilityDB/src/ttext_textfuncs.c' Lines executed:100.00% of 64 Creating '#home#travis#build#MobilityDB#MobilityDB#build#CMakeFiles#MobilityDB.dir#src#ttext_textfuncs.c.o###home#travis#build#MobilityDB#MobilityDB#src#ttext_textfuncs.c.gcov' [...]

with the most recent one

https://app.travis-ci.com/github/MobilityDB/MobilityDB/jobs/526613557

after_success 2.34s$ [ "$COVERAGE" == "1" ] && coveralls --exclude cmake --exclude doc --exclude include --exclude sql --exclude src --exclude test --exclude tools --gcov-options '-lp' -r .. -b . /home/travis/build/MobilityDB/MobilityDB/build/CMakeFiles/MobilityDB.dir/src/tnumber_spgist.c.gcda:cannot open data file, assuming not executed File '/home/travis/build/MobilityDB/MobilityDB/src/tnumber_spgist.c' No executable lines Removing '#home#travis#build#MobilityDB#MobilityDB#build#CMakeFiles#MobilityDB.dir#src#tnumber_spgist.c.o###home#travis#build#MobilityDB#MobilityDB#src#tnumber_spgist.c.gcov' [...]

there is an error reported. Any idea why it is the case ?

[estebanzimanyi]

I have tried an alternative approach and used Github actions https://github.com/estebanzimanyi/MobilityDB/blob/develop/.github/workflows/pgversion.yml issuing a similar command as in the previous .travis.yml file https://github.com/estebanzimanyi/MobilityDB/blob/develop/.travis.yml The run https://github.com/estebanzimanyi/MobilityDB/runs/3163152108?check_suite_focus=true generates an error

{'message': "Couldn't find a repository matching this job.", 'error': True}
Error: Process completed with exit code 1.

Any idea how I can solve this error ?

[cvvergara]

In here you can put [0,1] https://github.com/estebanzimanyi/MobilityDB/blob/develop/.github/workflows/pgversion.yml#L26 Because you have the if here: https://github.com/estebanzimanyi/MobilityDB/blob/develop/.github/workflows/pgversion.yml#L97 So that step will only execute when the matrix.coverage is = 1

[estebanzimanyi]

The coverage should run only once for the latest versions.

If I put [0,1] wouldn't this result in running the coverage tests for all the matrix combinations ? This is not necessary.

On Mon, Jul 26, 2021 at 6:51 PM Vicky Vergara @.***> wrote:

In here you can put [0,1]

https://github.com/estebanzimanyi/MobilityDB/blob/develop/.github/workflows/pgversion.yml#L26 Because you have the if here:

https://github.com/estebanzimanyi/MobilityDB/blob/develop/.github/workflows/pgversion.yml#L97 So that step will only execute when the matrix.coverage is = 1

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lemurheavy/coveralls-public/issues/1567#issuecomment-886864523, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGNHMBTRR755Y4CUCLY6N7TTZWHAPANCNFSM5AZS2C7A .

[afinetooth]

Dear James As you rightly pointed out, there are many changes since we are organizing the files and the directories in anticipation of a future release. Therefore, instead of trying to analyze where things are blocked, maybe the easiest way would be to "reinitialize" the repository with the new structure. Is there any document/resource that we can read to enable this kind of operation? Regards Esteban

Hi @estebanzimanyi, the simplest thing to do would be:

1. Log into https://coveralls.io
2. Go to ADD REPOS
3. Toggle OFF your repo (That "removes" it from our system / archives it)
4. Toggle the repo back ON (That reinitializes it with a fresh repo record.)

Let me know if you have any issues with that.

[afinetooth]

Comparying the output of the last coverage that worked [...] with the most recent one [...] there is an error reported. Any idea why it is the case ?

Only to say that, since the error is:

cannot open data file, assuming not executed

It looks like your integration can't find (or open) the coverage report file in the second run.

This:

File '/home/travis/build/MobilityDB/MobilityDB/src/tnumber_spgist.c'
Lines executed:87.12% of 264

Compared to this:

/home/travis/build/MobilityDB/MobilityDB/build/CMakeFiles/MobilityDB.dir/src/tnumber_spgist.c.gcda:cannot open data file, assuming not executed

Indicates the file location has changed, because your source directories have changed.

In build 2, you have a new set of source directories under:

/home/travis/build/MobilityDB/MobilityDB/

Specifically:

[...] MobilityDB/build/CMakeFiles/MobilityDB.dir/src/

Any idea why that happened?

[afinetooth]

I have tried an alternative approach and used Github actions https://github.com/estebanzimanyi/MobilityDB/blob/develop/.github/workflows/pgversion.yml

[...] generates an error

{'message': "Couldn't find a repository matching this job.", 'error': True}
Error: Process completed with exit code 1.

Any idea how I can solve this error ?

Yes, this error indicates the wrong identifier for the repo.

It doesn't look like you're using the official Coveralls Github Action. (Although you could, and it might be your best next step---just follow the usage instructions there.)

That said, if you are not using the Coveralls Github Action and you're sending over any value besides github for the service_name parameter in your JSON (See Coveralls API Reference), then coveralls will be looking for the repo's COVERALLS_REPO_TOKEN, rather than its ${{ secrets.GITHUB_TOKEN }}, as it's identifier.

You have two options:

1. Keep your current setup. Add a secret env var on the repo called COVERALLS_REPO_TOKEN and make that the value you pass for COVERALLS_REPO_TOKEN in your action's config yaml.
2. Use the Coveralls Github Action and, perhaps, not change anything with your current config yaml. (You'll have to run it and see.)

---

Update: It looks like you may have done this already, per this other version of your config yaml: https://github.com/estebanzimanyi/MobilityDB/blob/develop/.github/workflows/pgversion.yml#L97

Any luck there?

[cvvergara]

Hi, @afinetooth I will try the:

2. Use the Coveralls Github Action and, perhaps, not change anything with your current config yaml. (You'll have to run it and see.)

after I finish the changes I am making, because there are a lot pending. I am reorganizing the repository, I think its better to try it out after all the reorganization is finished. Basically when this issue is closed: https://github.com/MobilityDB/MobilityDB/issues/85 our related issue to this topic is: https://github.com/MobilityDB/MobilityDB/issues/98

[estebanzimanyi]

Many thanks James for your suggestion. I was able to use Coveralls Github Action (your option 2) ! https://github.com/estebanzimanyi/MobilityDB/runs/3165768201?check_suite_focus=true

[estebanzimanyi]

Since we obtained a coverage of 70% using the Coveralls Github Action while using cpp-coveralls previously we obtained 96% of coverage, I wanted to test option 1) you mentioned. I went to my account in Coveralls, generate a token and I put the value I obtained as an Actions secret in Github. Now the step in the yaml file looks as follows

      - name: Generate coverage
        if: matrix.coverage == '1'
        env:
          COVERALLS_REPO_TOKEN: "${{ secrets.COVERALLS_TOKEN }}"
        run: |
          cd build
          coveralls --include npoint --include point --include src --gcov-options '\-lp' -r .. -b .

The error message in the run https://github.com/estebanzimanyi/MobilityDB/runs/3169900390?check_suite_focus=true is as follows

{'message': "Couldn't find a repository matching this job.", 'error': True}
Error: Process completed with exit code 1.

Am I doing something wrong ?

[afinetooth]

@estebanzimanyi glad to hear you had some success with the Coveralls Github Action (Option 2).

The implementation for Option 1 is off.

To clarify the instructions better, here are the steps:

1. Go to the coveralls repo page for your repo, assuming it's: https://coveralls.io/github/estebanzimanyi/MobilityDB
2. Copy the repo token provided on that page, under TOKEN, ex: UwjdO5MAYbMVBfiliE8Zz6uDeIVwwYbQj
3. Make that the value of your repo secret.
4. Try again.

I'm thinking you may have used a personal access token from your coveralls account page, here. That's actually different from the repo token, which you'll find here and here.

[estebanzimanyi]

Many thanks James for your help in this issue. Now we are able to use the Coveralls Github Action again for our project. https://github.com/MobilityDB/MobilityDB/pull/139

[afinetooth]

Glad to hear it, @estebanzimanyi. Thanks for the update!