search

hpi-swa / smalltalkCI

Framework for testing Smalltalk projects with GitHub Actions, GitLab CI, Travis CI, AppVeyor, and others.
MIT License
96 stars 68 forks source link

Coveralls shows 0% coverage for a matrix CI workflow on GitHub #536

Open LinqLover opened 3 years ago

LinqLover commented 3 years ago

Here is one of the erroneously calculated builds: https://coveralls.io/builds/40758673 However, all other current builds look identically. Note that the CI passes on GitHub and the local TestRunner in my image attests me a 50% coverage. In particular, note that in the past™ it already had worked: https://coveralls.io/builds/40745486

In that earlier build, I had fail-fast enabled and some jobs failing, but could this make a difference? Does smalltalkCI handle parallel runs or do I have to use the coverallsapp action manually as described in the docs?

LinqLover commented 3 years ago

PS: Is there an option to optimistically merge all coverage data for platform-specific code branches or would coveralls do this for me automatically? :)

fniephaus commented 3 years ago

I vaguely remember adding a parallel flag and that CI would need to tell coveralls specifically about the completion of a build unit: https://github.com/hpi-swa/smalltalkCI/blob/79bbab1f8542263731e1b99fd094242d79eef46e/helpers.sh#L422

I'm afraid I don't have any answers for your questions. If you could explore and improve the docs, that would be great.

LinqLover commented 3 years ago

I added the COVERALLS_PARALLEL to my build but this didn't help. I'm also confused because on this repository, the coverage has always been detected correctly ... Is this really non-deterministic behavior and based on the order of job execution? :thinking:

fniephaus commented 3 years ago

It's very much possible that the order is important. If you could debug this that'd be great!

gcotelli commented 3 years ago

For coveralls I'm using the official GitHub action and see no problems with the coverage reporting. Take a look for example at https://github.com/ba-st/Launchpad/blob/release-candidate/.github/workflows/unit-tests.yml and https://github.com/ba-st/Launchpad/blob/release-candidate/.smalltalkci/.unit-tests.ston

On Tue, Oct 26, 2021, 08:49 Fabio Niephaus @.***> wrote:

It's very much possible that the order is important. If you could debug this that'd be great!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hpi-swa/smalltalkCI/issues/536#issuecomment-951855189, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQCK6BNNLDZHAA75PZI7H3UI2PUNANCNFSM47CLCJPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.