codecov / codecov-action

GitHub Action that uploads coverage to Codecov :open_umbrella:
https://www.codecov.io
MIT License
1.45k stars 205 forks source link

[v4] Error: Write EPIPE #1280

Closed houserx-jmcc closed 4 months ago

houserx-jmcc commented 7 months ago

When switching from a working v3 setup to the new v4, we encounter this error 10-20% of the time in a high volume monorepo. Re-runs succeed on the 2nd or 3rd try. We do host our own GitHub runners.

Also a quick shoutout for https://github.com/codecov/codecov-action/issues/926 - if we had builtin retries this would be much less of a problem.

==> linux OS detected
https://cli.codecov.io/latest/linux/codecov.SHA256SUM
Received SHA256SUM 103bfefcc56f76473179e600b96eb8150b0f349ad94836b0f63f03ffac469ad7  codecov
Received SHA256SUM signature -----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEJwNOf9uFDgu8LGL/gGuyiu13mGkFAmW8+QUACgkQgGuyiu13
mGll6A/8CVDQ3LQaqpgQEIekCJUNKU/+tpF1EGpuUsMi7DZZ0RA6cmazUJFUlZK2
GS7Ptcwly+SYTT+a61SzeVlewPLa47XZIFyC+7tgSO6XgUNlJ+KTevyeL6GG0Qs1
YaciuJv8IHAqaaYT49iKpWE99OME1VXY3fIm4uEHlc4pgnmLv+FNdxnit4AYLGlw
2JHzdDMd5aHlnYYIyYJ9UbM8fFVWgddL1venoYl59NKc5gXjH1/+yPPWz2R+4f[22](https://github.com/houserx/micro-services/actions/runs/7832462490/job/21371233166#step:15:23)
jZfofI04aEJxNhGinfV5Vykb9asyfMupmLUweArgTIF3wzIEoYo/pK0nVgRBiROE
1hiaH5lti8brA2NF+pzp4+xFEyCU1m/mgN/rj1VRkDs0CW4S86eYWVbfuHEb0ymQ
o7Oe/rST6IjqM72B8eleEdT1DKdeX8DYSXnKvR2J1POyoPMMt/HUoCxphiNOq6Ei
416xOCgqWwOEGbeZ1pxp4Eovf6fffbd2F9MUcJTzgWocOqLh3lZB/EX3G9eLKsyf
WLW3s3NXzajS9j4zEPZxGPmnqAadfm9dzffwokFZnqMTJ7HB4qXH87BPgwPbq8R+
y/1Hv7hHEbJJhbE1fitwHFMg4gfUP6q39VtrfooQSBRYYDzrstowO/L2xsr+AkwW
EQGY70I4SY4XPKKzs8tF3Jch7Oa+xgeNMM1qSGyb2Vjn0KKSDMk=
=jEIQ
-----END PGP SIGNATURE-----

==> Running version latest
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: /home/runner/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB[28](https://github.com/houserx/micro-services/actions/runs/7832462490/job/21371233166#step:15:29)AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" imported
gpg: Total number processed: 1
gpg:               imported: 1

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:952:11)
    at Socket._write (node:net:964:8)
    at writeOrBuffer (node:internal/streams/writable:447:12)
    at _write (node:internal/streams/writable:389:10)
    at Socket.Writable.end (node:internal/streams/writable:665:17)
    at Socket.end (node:net:722:[31](https://github.com/houserx/micro-services/actions/runs/7832462490/job/21371233166#step:15:32))
    at module.exports (/runner/_work/_actions/codecov/codecov-action/v4/node_modules/gpg/lib/spawnGPG.js:[50](https://github.com/houserx/micro-services/actions/runs/7832462490/job/21371233166#step:15:51):1)
    at Object.call (/runner/_work/_actions/codecov/codecov-action/v4/node_modules/gpg/lib/gpg.js:28:1)
Jolg42 commented 7 months ago

Same here, I actually created an issue a few seconds after you 😄 https://github.com/codecov/codecov-action/issues/1281

drazisil-codecov commented 7 months ago

@rohan-at-sentry can you help @thomasrockhu-codecov get this investigated and resolved? Sound potentially CLI-related.

pat-kasprzyk-kiefa commented 7 months ago

I am also experiencing this error on my mono-repo, let me know what I can do to help investigate

thomasrockhu-codecov commented 7 months ago

@houserx-jmcc can you share a link to CI where this is failing? or the config file specifying the os and hardware?

houserx-jmcc commented 7 months ago

@thomasrockhu-codecov We can't provide a link as it is an internal/private repository, but we are using the vanilla image from actions-runner-controller for self-hosting GitHub runners on AWS EKS.

crazy-max commented 6 months ago

Here is a repro: https://github.com/moby/buildkit/actions/runs/8092979752/job/22114788142?pr=4717#step:9:34

houserx-jmcc commented 6 months ago

Is this no longer being actively worked on?

thomasrockhu-codecov commented 6 months ago

@houserx-jmcc no it is, sorry I was doing some cleanup on my side

thomasrockhu-codecov commented 5 months ago

@houserx-jmcc sorry for the delay here. I implemented retries and wanted to see if this will work for you.

In your CI, would you be able to replace the action with

uses: codecov/codecov-action@d1f1eb8ced025a661e3a2070d7c9462978291164

houserx-jmcc commented 5 months ago

2 days without seeing the error using this version. I'll report back if we see it, but assume no news is good news otherwise.

houserx-jmcc commented 5 months ago

I spoke too soon:

image

Jolg42 commented 4 months ago

My colleague Alberto spotted one error today, happening on our public repo: https://github.com/prisma/prisma/actions/runs/8893795874/job/24420825519?pr=24029#step:6:30

Run codecov/codecov-action@v4
  with:
    token: ***
    files: ./packages/client/src/__tests__/coverage/clover.xml
    flags: client,ubuntu,driveradapters
    name: client-ubuntu-driveradapters
  env:
    HAS_BUILDPULSE_SECRETS: true
    PRISMA_TELEMETRY_INFORMATION: prisma test.yml
    PRISMA_HIDE_UPDATE_MESSAGE: true
    PNPM_HOME: /home/runner/setup-pnpm/node_modules/.bin
    NPM_CONFIG_USERCONFIG: /home/runner/work/_temp/.npmrc
    NODE_AUTH_TOKEN: XXXXX-XXXXX-XXXXX-XXXXX
==> linux OS detected
https://cli.codecov.io/latest/linux/codecov.SHA256SUM
==> Running version latest
==> Running version v0.5.2
==> Running git config --global --add safe.directory /home/runner/work/prisma/prisma
/usr/bin/git config --global --add safe.directory /home/runner/work/prisma/prisma
==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov create-commit'
/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov create-commit --git-service github -C 151b144bf019d806b1282aae486ec97005ab97e6
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: /home/runner/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" imported
gpg: Total number processed: 1
gpg:               imported: 1

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:952:11)
    at Socket._write (node:net:964:8)
    at writeOrBuffer (node:internal/streams/writable:447:12)
    at _write (node:internal/streams/writable:389:10)
    at Socket.Writable.end (node:internal/streams/writable:665:17)
    at Socket.end (node:net:722:31)
    at module.exports (/home/runner/work/_actions/codecov/codecov-action/v4/node_modules/gpg/lib/spawnGPG.js:50:1)
    at Object.call (/home/runner/work/_actions/codecov/codecov-action/v4/node_modules/gpg/lib/gpg.js:28:1)
thomasrockhu-codecov commented 4 months ago

Got it @houserx-jmcc @Jolg42, to be clear this not an always thing (for reproducibility)?

houserx-jmcc commented 4 months ago

@thomasrockhu-codecov correct, only happening sometimes. 1-2 retries of the same job in a workflow will usually succeed if the first time failed.

thomasrockhu-codecov commented 4 months ago

@houserx-jmcc got it, yeah, let me work on implementing retries for this

Jolg42 commented 4 months ago

@thomasrockhu-codecov It's flaky, only happens once in a while and retrying usually works but since retrying means a human needs to retry and then wait, it's sub-otimal. An automatic retry would be much better for this.

thomasrockhu-codecov commented 4 months ago

@houserx-jmcc would you be able to try 6a309e2c1e1a37a02b42e5b83e33e8b7b4e5efa8?

houserx-jmcc commented 4 months ago

It seems a bit less frequent, but the issue is still occurring. Example output from 6a309e2:

Run codecov/codecov-action@6a309e2c1e1a37a02b42e5b83e33e8b7b4e5efa8
==> linux OS detected
https://cli.codecov.io/latest/linux/codecov.SHA256SUM
Received SHA256SUM e70beb7c9e3d894678e7d4d0fcb94e59[13](https://github.com/houserx/micro-services/actions/runs/8973546064/job/24643992188#step:12:14)3212dbda5ca7406b625a0167ce4ca8  codecov
Received SHA256SUM signature -----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEJwNOf9uFDgu8LGL/gGuyiu13mGkFAmYhSq4ACgkQgGuyiu13
mGkULA/8Dq6mYMCj4DxoHtfQeiQ3FZ/8HrgQmwUICQa5QHatbojQLmsYkruvvWWl
+Q7PywoiEWkFQ6nI1Wb4sTLWAMwdjjCKSECqTs9jlqXklKDKyb1Fv5O0ytwFEe/a
Wc4V4CKvH3LFHWB/ZsFMmjBBLQ3evQqz5rfk+Bih5uV/RtcNoLnWstaIqyeISyb4
NdgLWy9uapyeHfoPoZ6HEdrNhy4LvMrlCbgjwjFLZ445fE0Neq4n3QOlDKZ+RviO
FU/zYlZkWyHxrDavBEotbSWRCNB/ptyTc9uuQXEtemrJKoJYNRAIBx7N60uEEpCO
oDzALNV404Tw7WiULHyUhuxeN4EDiu2mCoa1KiMjFHjOayY6lREtsvt94ubGmAJ1
y29LiEvYIxEzd7nJcC8PzxbZk6AUv3IXA8o9m47KCVqKFMCFCTgc8fEWlrIfRtLN
HupcoOOuV7SWHXfg5vOf0vclnTpw3MVcrj+Z7dMA5OSpqcTlL8ACb9QPsKpAesUR
tth7zorPoU/RHfgEgKNPmKX9vvqB+AZ2n2oCPhlYOBP6/9NFjoIvIfWZPD9FSce7
K4FOEyaBEyypUMXtjPVj8MtoOaLk/X9g03ZKAlDxbnBIqqp0YLbqMW+nZkG8TTKI
e4d6XX447TyQ2750658NX+9Z/0c/oK+jsNCG2uocOkIL3kOHtAw=
=QUC9
-----END PGP SIGNATURE-----

==> Running version latest
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: /home/runner/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" imported
gpg: Total number processed: 1
gpg:               imported: 1

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:[15](https://github.com/houserx/micro-services/actions/runs/8973546064/job/24643992188#step:12:16))
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:952:11)
    at Socket._write (node:net:964:8)
    at writeOrBuffer (node:internal/streams/writable:447:12)
    at _write (node:internal/streams/writable:389:10)
    at Socket.Writable.end (node:internal/streams/writable:665:[17](https://github.com/houserx/micro-services/actions/runs/8973546064/job/24643992188#step:12:18))
    at Socket.end (node:net:722:31)
    at module.exports (/runner/_work/_actions/codecov/codecov-action/6a309e2c1e1a37a02b42e5b83e33e8b7b4e5efa8/node_modules/gpg/lib/spawnGPG.js:50:1)
    at Object.call (/runner/_work/_actions/codecov/codecov-action/6a309e2c1e1a37a02b42e5b83e33e8b7b4e5efa8/node_modules/gpg/lib/gpg.js:28:1)
thomasrockhu-codecov commented 4 months ago

@houserx-jmcc blast. I'll keep on noodling

thomasrockhu-codecov commented 4 months ago

Hi all, I think this should be solved in 4.4.0 now. Please let me know if that's not the case