coverallsapp / orb

Coveralls CircleCI Orb
https://circleci.com/orbs/registry/orb/coveralls/coveralls
MIT License
2 stars 16 forks source link

fix: Ensure that `coveralls report` command succeeds when `fail_on_error: false` is set #41

Closed patbl closed 4 weeks ago

patbl commented 7 months ago

Previously, if the coveralls.sh script fails to download the coverage reporter, running coveralls report resulted in a command-not-found error (error code 127). This failure once occurred multiple times over the course of a day for me, so it's not not just a theoretical concern.

Fixes https://github.com/coverallsapp/orb/issues/40.

afinetooth commented 4 weeks ago

@patbl thanks so much for your PR.

Your code worked fine, but we decided to make several different changes throughout the script, targeting individual setup tasks, that are parallel to changes we'll be making in our github-action, in this PR: https://github.com/coverallsapp/orb/pull/46

I performed manual tests of fail_on_error with success for these scenarios:

  1. Failure of download
  2. Failure to verify checksums
  3. Failure to find binary named coveralls

Example:

# Failure to find binary named `coveralls`
# (Removed file after extraction)
root@8da46491130b:/app# export COVERALLS_FAIL_ON_ERROR="1"
root@8da46491130b:/app# ./test_coveralls_script.sh
coveralls-linux.tar.gz: OK
Coveralls binary not found after extraction.
root@8da46491130b:/app# echo $?
1

root@8da46491130b:/app# export COVERALLS_FAIL_ON_ERROR="false"
root@8da46491130b:/app# ./test_coveralls_script.sh
coveralls-linux.tar.gz: OK
Coveralls binary not found after extraction.
root@8da46491130b:/app# echo $?
0

I intend to implement those tests (of "setup failures") as steps in our CI pipeline at some point. Just didn't have time to get there today.