Mono repository incorrectly reporting success

awentzel commented 4 years ago

Is your feature request related to a problem? Please describe. When running the following GitHub Action on a Lerna mono-repository if any single package fails, it still reports success.

In my example, the projectRoot incorrectly pointed to ../../../ instead of below code block, causing format_coverage to break, yet GH Action codeclimate-action to report success though really fail.

    "coverageReporters": [
      "json", 
      [
        "lcov", {"projectRoot": "../../"}
      ]
    ],

Describe the solution you'd like Accurately report status as success and see that through on the Code Climate settings test reporting pages.

Additional context Add any other context or screenshots about the feature request here.

Actual Error

Error: open workspace/sites/fast-component-explorer/app/app.tsx: no such file or directory
| Usage:
|   cc-test-reporter format-coverage [coverage file] [flags]
| 
| Flags:
|       --add-prefix string   add this prefix to file paths
|   -t, --input-type string   type of input source to use [clover, cobertura, coverage.py, excoveralls, gcov, gocov, jacoco, lcov, simplecov, xccov]
|   -o, --output string       output path (default "coverage/codeclimate.json")
|   -p, --prefix string       the root directory where the coverage analysis was performed (default "/github/workspace")
| 
| Global Flags:
|   -d, --debug   run in debug mode
| 
| (node:5413) UnhandledPromiseRejectionWarning: TypeError: (s || "").replace is not a function
|     at escapeData (/actions/paambaati-codeclimate-action@v2.6.0/node_modules/@actions/core/lib/command.js:66:10)
|     at Command.toString (/actions/paambaati-codeclimate-action@v2.6.0/node_modules/@actions/core/lib/command.js:60:35)
|     at issueCommand (/actions/paambaati-codeclimate-action@v2.6.0/node_modules/@actions/core/lib/command.js:23:30)
|     at Object.issue (/actions/paambaati-codeclimate-action@v2.6.0/node_modules/@actions/core/lib/command.js:27:5)
|     at Object.error (/actions/paambaati-codeclimate-action@v2.6.0/node_modules/@actions/core/lib/core.js:127:15)
|     at /actions/paambaati-codeclimate-action@v2.6.0/lib/main.js:135:28
|     at Generator.throw (<anonymous>)
|     at rejected (/actions/paambaati-codeclimate-action@v2.6.0/lib/main.js:6:65)
|     at processTicksAndRejections (internal/process/task_queues.js:97:5)
| (node:5413) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
| (node:5413) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
[CI - FAST Local Testing/build_local_linux]   ✅  Success - Collect and upload code coverage to Code Climate

wagenet commented 4 years ago

I'm seeing something similar though my scenario is a bit simpler. Here are my logs:

2020-05-06T22:52:09.8261414Z [command]/__w/skylight-rust/skylight-rust/cc-reporter format-coverage /__w/skylight-rust/skylight-rust/coverage/lcov.info -t lcov -o codeclimate.0.json
2020-05-06T22:52:09.8338360Z fatal: Not a git repository (or any parent up to mount point /__w)
2020-05-06T22:52:09.8339052Z Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2020-05-06T22:52:09.8342532Z Error: exit status 128
2020-05-06T22:52:09.8352365Z Usage:
2020-05-06T22:52:09.8352933Z   cc-test-reporter format-coverage [coverage file] [flags]
2020-05-06T22:52:09.8353035Z 
2020-05-06T22:52:09.8353169Z Flags:
2020-05-06T22:52:09.8353469Z       --add-prefix string   add this prefix to file paths
2020-05-06T22:52:09.8353874Z   -t, --input-type string   type of input source to use [clover, cobertura, coverage.py, excoveralls, gcov, gocov, jacoco, lcov, simplecov, xccov]
2020-05-06T22:52:09.8354218Z   -o, --output string       output path (default "coverage/codeclimate.json")
2020-05-06T22:52:09.8354631Z   -p, --prefix string       the root directory where the coverage analysis was performed (default "/__w/skylight-rust/skylight-rust")
2020-05-06T22:52:09.8354743Z 
2020-05-06T22:52:09.8354875Z Global Flags:
2020-05-06T22:52:09.8355144Z   -d, --debug   run in debug mode
2020-05-06T22:52:09.8355229Z 
2020-05-06T22:52:09.8379983Z (node:823) UnhandledPromiseRejectionWarning: TypeError: (s || "").replace is not a function
2020-05-06T22:52:09.8380659Z     at escapeData (/__w/_actions/paambaati/codeclimate-action/v2.6.0/node_modules/@actions/core/lib/command.js:66:10)

pierre-emmanuelJ commented 4 years ago

I got something similar with Go coverage.

- name: Publish code coverage
   uses: paambaati/codeclimate-action@v2.6.0
   env:
     CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
   with:
     coverageCommand: go test -race -coverprofile=c.out -covermode=atomic ./...
     debug: true

2020-05-20T09:53:06.6370719Z ##[group]Run paambaati/codeclimate-action@v2.6.0
2020-05-20T09:53:06.6370961Z with:
2020-05-20T09:53:06.6371241Z   coverageCommand: go test -race -coverprofile=c.out -covermode=atomic ./...
2020-05-20T09:53:06.6371561Z   debug: true
2020-05-20T09:53:06.6371857Z env:
2020-05-20T09:53:06.6371992Z   GOROOT: /opt/hostedtoolcache/go/1.14.3/x64
2020-05-20T09:53:06.6372135Z   CC_TEST_REPORTER_ID: 
2020-05-20T09:53:06.6372224Z ##[endgroup]
2020-05-20T09:53:07.0952012Z [command]/home/runner/work/egoscale/egoscale/cc-reporter before-build
2020-05-20T09:53:07.1063600Z [command]/opt/hostedtoolcache/go/1.14.3/x64/bin/go test -race -coverprofile=c.out -covermode=atomic ./...
2020-05-20T09:53:27.9971384Z ok     github.com/exoscale/egoscale    15.261s coverage: 74.2% of statements
2020-05-20T09:53:27.9976440Z ?      github.com/exoscale/egoscale/admin  [no test files]
2020-05-20T09:53:27.9977016Z ?      github.com/exoscale/egoscale/generate   [no test files]
2020-05-20T09:53:27.9977387Z ok     github.com/exoscale/egoscale/internal/v2    0.053s  coverage: 0.7% of statements
2020-05-20T09:53:28.0083020Z [command]/home/runner/work/egoscale/egoscale/cc-reporter after-build --exit-code 0 --debug
2020-05-20T09:53:28.0148833Z time="2020-05-20T09:53:28Z" level=debug msg="about to run format-coverage" 
2020-05-20T09:53:28.0149626Z time="2020-05-20T09:53:28Z" level=debug msg="searching for a formatter to use" 
2020-05-20T09:53:28.0150210Z time="2020-05-20T09:53:28Z" level=debug msg="checking clover formatter" 
2020-05-20T09:53:28.0150794Z time="2020-05-20T09:53:28Z" level=debug msg="checking search path build/logs/clover.xml for clover formatter" 
2020-05-20T09:53:28.0151529Z time="2020-05-20T09:53:28Z" level=debug msg="checking search path clover.xml for clover formatter" 
2020-05-20T09:53:28.0152525Z time="2020-05-20T09:53:28Z" level=debug msg="checking cobertura formatter" 
2020-05-20T09:53:28.0153256Z time="2020-05-20T09:53:28Z" level=debug msg="checking search path cobertura.xml for cobertura formatter" 
2020-05-20T09:53:28.0154204Z time="2020-05-20T09:53:28Z" level=debug msg="checking excoveralls formatter" 
2020-05-20T09:53:28.0155137Z time="2020-05-20T09:53:28Z" level=debug msg="checking search path cover/excoveralls.json for excoveralls formatter" 
2020-05-20T09:53:28.0155778Z time="2020-05-20T09:53:28Z" level=debug msg="checking coverage.py formatter" 
2020-05-20T09:53:28.0157014Z time="2020-05-20T09:53:28Z" level=debug msg="checking search path coverage.xml for coverage.py formatter" 
2020-05-20T09:53:28.0157472Z time="2020-05-20T09:53:28Z" level=debug msg="checking gcov formatter" 
2020-05-20T09:53:28.0157831Z time="2020-05-20T09:53:28Z" level=debug msg="checking search path ./ for GCov formatter" 
2020-05-20T09:53:28.0158209Z time="2020-05-20T09:53:28Z" level=debug msg="checking gocov formatter" 
2020-05-20T09:53:28.0158583Z time="2020-05-20T09:53:28Z" level=debug msg="checking search path c.out for gocov formatter" 
2020-05-20T09:53:28.0159520Z time="2020-05-20T09:53:28Z" level=debug msg="found file c.out for gocov formatter" 
2020-05-20T09:53:29.4944524Z time="2020-05-20T09:53:29Z" level=debug msg="couldn't load committed at from ENV, trying git..." 
2020-05-20T09:53:29.5394028Z time="2020-05-20T09:53:29Z" level=info msg="trimming with prefix /home/runner/work/egoscale/egoscale/" 
2020-05-20T09:53:29.5397018Z time="2020-05-20T09:53:29Z" level=debug msg="getting fallback blob_id for source file github.com/exoscale/egoscale/accounts.go" 
2020-05-20T09:53:29.5397514Z time="2020-05-20T09:53:29Z" level=error msg="failed to read file github.com/exoscale/egoscale/accounts.go\nopen github.com/exoscale/egoscale/accounts.go: no such file or directory" 
2020-05-20T09:53:29.5397731Z Error: open github.com/exoscale/egoscale/accounts.go: no such file or directory
2020-05-20T09:53:29.5400760Z Usage:
2020-05-20T09:53:29.5401413Z   cc-test-reporter after-build [flags]
2020-05-20T09:53:29.5401669Z 
2020-05-20T09:53:29.5401940Z Flags:
2020-05-20T09:53:29.5402449Z   -s, --batch-size int               batch size for source files (default 500)
2020-05-20T09:53:29.5403411Z   -e, --coverage-endpoint string     endpoint to upload coverage information to (default "https://api.codeclimate.com/v1/test_reports")
2020-05-20T09:53:29.5404055Z   -t, --coverage-input-type string   type of input source to use [clover, cobertura, coverage.py, excoveralls, gcov, gocov, jacoco, lcov, simplecov, xccov]
2020-05-20T09:53:29.5404539Z       --exit-code int                exit code of the test run
2020-05-20T09:53:29.5405036Z   -r, --id string                    reporter identifier
2020-05-20T09:53:29.5405452Z       --insecure                     send coverage insecurely (without HTTPS)
2020-05-20T09:53:29.5405999Z   -p, --prefix string                the root directory where the coverage analysis was performed (default "/home/runner/work/egoscale/egoscale")
2020-05-20T09:53:29.5406272Z 
2020-05-20T09:53:29.5406537Z Global Flags:
2020-05-20T09:53:29.5406888Z   -d, --debug   run in debug mode
2020-05-20T09:53:29.5407114Z 
2020-05-20T09:53:29.5425744Z (node:3245) UnhandledPromiseRejectionWarning: TypeError: (s || "").replace is not a function
2020-05-20T09:53:29.5426661Z     at escapeData (/home/runner/work/_actions/paambaati/codeclimate-action/v2.6.0/node_modules/@actions/core/lib/command.js:66:10)
2020-05-20T09:53:29.5427258Z     at Command.toString (/home/runner/work/_actions/paambaati/codeclimate-action/v2.6.0/node_modules/@actions/core/lib/command.js:60:35)
2020-05-20T09:53:29.5427818Z     at issueCommand (/home/runner/work/_actions/paambaati/codeclimate-action/v2.6.0/node_modules/@actions/core/lib/command.js:23:30)
2020-05-20T09:53:29.5428405Z     at Object.issue (/home/runner/work/_actions/paambaati/codeclimate-action/v2.6.0/node_modules/@actions/core/lib/command.js:27:5)
2020-05-20T09:53:29.5429179Z     at Object.error (/home/runner/work/_actions/paambaati/codeclimate-action/v2.6.0/node_modules/@actions/core/lib/core.js:127:15)
2020-05-20T09:53:29.5429735Z     at /home/runner/work/_actions/paambaati/codeclimate-action/v2.6.0/lib/main.js:189:20
2020-05-20T09:53:29.5430084Z     at Generator.throw (<anonymous>)
2020-05-20T09:53:29.5430559Z     at rejected (/home/runner/work/_actions/paambaati/codeclimate-action/v2.6.0/lib/main.js:6:65)
2020-05-20T09:53:29.5430861Z     at processTicksAndRejections (internal/process/task_queues.js:93:5)
2020-05-20T09:53:29.5431170Z (node:3245) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
2020-05-20T09:53:29.5431857Z (node:3245) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

paambaati commented 4 years ago

Can you folks try out the latest v2.7.1 release? It has better error handling and should help you pinpoint issues better.

KillyMXI commented 2 years ago

I think I got a very related issue.

coverage action:

      - name: Publish coverage results
        uses: paambaati/codeclimate-action@v2.7.5
        env:
          CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
        with:
          coverageCommand: npm run cover
          coverageLocations: |
            ${{github.workspace}}/packages/html-to-text/coverage/lcov.info:lcov
          debug: true

Error report:

/home/runner/work/node-html-to-text/node-html-to-text/cc-reporter format-coverage 
/home/runner/work/node-html-to-text/node-html-to-text/packages/html-to-text/coverage/lcov.info -t lcov -o codeclimate.0.json --debug
time="2021-11-14T17:16:46Z" level=debug msg="coverage path /home/runner/work/node-html-to-text/node-html-to-text/packages/html-to-text/coverage/lcov.info" 
time="2021-11-14T17:16:46Z" level=debug msg="using formatter lcov" 
time="2021-11-14T17:16:46Z" level=debug msg="checking search path /home/runner/work/node-html-to-text/node-html-to-text/packages/html-to-text/coverage/lcov.info for lcov formatter" 
time="2021-11-14T17:16:48Z" level=debug msg="couldn't load committed at from ENV, trying git..." 
time="2021-11-14T17:16:48Z" level=info msg="trimming with prefix /home/runner/work/node-html-to-text/node-html-to-text/" 
time="2021-11-14T17:16:48Z" level=debug msg="getting fallback blob_id for source file src/formatter.js" 
time="2021-11-14T17:16:48Z" level=error msg="failed to read file src/formatter.js\nopen src/formatter.js: no such file or directory" 
Error: open src/formatter.js: no such file or directory

There is a misformed message in the next to last line:

msg="failed to read file src/formatter.js\nopen src/formatter.js: no such file or directory"

But what I think is the actual cause of the issue: The action seems to be looking for source files from the root or the repository. This is just not true for a monorepo. And I can't provide a custom prefix because it has to be different for each package. That is, each coverage location has it's own prefix.

paambaati / codeclimate-action

Mono repository incorrectly reporting success #169