nodejs / reliability

Make the CI green again
35 stars 11 forks source link

CI Reliability 2024-05-03 #852

Open github-actions[bot] opened 4 months ago

github-actions[bot] commented 4 months ago

Failures in node-test-pull-request/58781 to node-test-pull-request/58876 that failed 2 or more PRs (Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2024-05-03 00:08 5 1 23 18 53 1.30%

Jenkins Failure

Reason Build timed out (after 60 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52714/, https://github.com/nodejs/node/pull/52751/)
Appeared test-ibm-ubi81_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58800/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58809/
Example ``` Build timed out (after 60 minutes). Marking the build as failed. ```

Git Failure

Build Failure

Reason error C1060: compiler is out of heap space [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj]
Type BUILD_FAILURE
Failed PR 10 (https://github.com/nodejs/node/pull/52627/, https://github.com/nodejs/node/pull/52587/, https://github.com/nodejs/node/pull/52760/, https://github.com/nodejs/node/pull/52744/, https://github.com/nodejs/node/pull/52781/, https://github.com/nodejs/node/pull/52646/, https://github.com/nodejs/node/pull/52794/, https://github.com/nodejs/node/pull/52802/, https://github.com/nodejs/node/pull/52810/, https://github.com/nodejs/node/pull/52766/)
Appeared test-rackspace-win2022_vs2022-x64-5, test-rackspace-win2022_vs2022-x64-6, test-rackspace-win2022_vs2022-x64-4, test-rackspace-win2022_vs2022-x64-3, test-rackspace-win2022_vs2022-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58802/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58876/
Example ``` error C1060: compiler is out of heap space [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj] Creating library ..\..\out\Release\mksnapshot_host.lib and object ..\..\out\Release\mksnapshot_host.exp setup-heap-internal.cc C:\workspace\node-compile-windows\node\deps\v8\src\heap\setup-heap-internal.cc(681): fatal error C1002: compiler is out of heap space in pass 2 [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj] growable-fixed-array-tq-csa.cc ```

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/52751/, https://github.com/nodejs/node/pull/52545/, https://github.com/nodejs/node/pull/52766/, https://github.com/nodejs/node/pull/52609/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-4, test-ibm-ubuntu2204_sharedlibs_container-x64-2, test-digitalocean-ubuntu2204_sharedlibs_container-x64-10
First CI https://ci.nodejs.org/job/node-test-pull-request/58806/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58845/
Example ``` fatal: No rebase in progress? ```

Reason ERROR: No files found with the specified search criteria.
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52751/, https://github.com/nodejs/node/pull/52545/, https://github.com/nodejs/node/pull/52706/)
Appeared test-rackspace-win2022_vs2022-x64-6, test-rackspace-win2022_vs2022-x64-5, test-rackspace-win2022_vs2022-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58806/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58817/
Example ``` ERROR: No files found with the specified search criteria. 1 file(s) copied. > set DEBUG_HELPER=1 > call vcbuild.bat arm64 release msi ```

Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52781/, https://github.com/nodejs/node/pull/52793/, https://github.com/nodejs/node/pull/52794/)
Appeared test-equinix-ubuntu1804_sharedlibs_container-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58866/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58869/
Example ``` ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? Collecting metadata... Metadata collection done. Notifying upstream projects of job completion Finished: FAILURE ```

Reason Failed to trigger sub builds
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51575/, https://github.com/nodejs/node/pull/52583/, https://github.com/nodejs/node/pull/50992/)
Appeared undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/58783/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58873/
Example ``` Failed to trigger sub builds ```

Reason error: remote GitHub already exists.
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52781/, https://github.com/nodejs/node/pull/52646/)
Appeared test-rackspace-win2019_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/58866/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58867/
Example ``` error: remote GitHub already exists. > git fetch --prune --no-tags GitHub > git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp ```

CCTest Failure

JSTest Failure

Reason parallel/test-http2-large-write-multiple-requests
Type JS_TEST_FAILURE
Failed PR 7 (https://github.com/nodejs/node/pull/52747/, https://github.com/nodejs/node/pull/52767/, https://github.com/nodejs/node/pull/52609/, https://github.com/nodejs/node/pull/52763/, https://github.com/nodejs/node/pull/52781/, https://github.com/nodejs/node/pull/52760/, https://github.com/nodejs/node/pull/52810/)
Appeared test-digitalocean-ubuntu2204_sharedlibs_container-x64-5, test-azure_msft-win11_vs2022-x64-2, test-ibm-rhel8-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-7, test-ibm-ubi81_container-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-2, test-digitalocean-ubuntu2204_sharedlibs_container-x64-6, test-ibm-rhel8-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/58796/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58874/
Example ``` not ok 2828 parallel/test-http2-large-write-multiple-requests --- duration_ms: 120029.05500 severity: fail exitcode: -15 stack: |- timeout server sends content 1 server sends content 2 server sends content 3 server sends content 4 server sends content 5 server sends content 6 server sends content 7 server sends content 8 server sends content 9 server sends content 10 server sends content 11 server sends content 12 server sends content 13 server sends content 14 server sends content 15 server sends content 16 server sends content 17 server sends content 18 server sends content 19 server sends content 20 server sends content 21 server sends content 22 server sends content 23 server sends content 24 server sends content 25 server sends content 26 server sends content 27 server sends content 28 server sends content 29 server sends content 30 server sends content 31 server s... ```

Reason benchmark/test-benchmark-crypto
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/52739/, https://github.com/nodejs/node/pull/52728/, https://github.com/nodejs/node/pull/52347/, https://github.com/nodejs/node/pull/52747/, https://github.com/nodejs/node/pull/52646/, https://github.com/nodejs/node/pull/52763/)
Appeared test-digitalocean-ubuntu2204_sharedlibs_container-x64-10, test-digitalocean-fedora37-x64-1, test-equinix-rhel8_container-arm64-1, test-ibm-rhel9-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-5, test-digitalocean-debian12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58785/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58834/
Example ``` not ok 4013 benchmark/test-benchmark-crypto --- duration_ms: 10483.62700 severity: fail exitcode: 1 stack: |- node:assert:126 throw new AssertionError(obj); ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 1 !== 0 at main (/home/iojs/build/workspace/node-test-commit-linux-containered/benchmark/crypto/timingSafeEqual.js:21:10) at /home/iojs/build/workspace/node-test-commit-linux-containered/benchmark/common.js:54:9 at process.processTicksAndRejections (node:internal/process/task_queues:77:11) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 1, expected: 0, operator: 'strictEqual' } Node.js v23.0.0-pre node:assert:126 throw new AssertionError(obj); ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 1 !== 0 at ChildProcess. (/home/iojs/build/workspace/node-test-commit-linux-containered/test... ```

Reason parallel/test-net-write-fully-async-buffer
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/52646/, https://github.com/nodejs/node/pull/52763/, https://github.com/nodejs/node/pull/52768/, https://github.com/nodejs/node/pull/52781/, https://github.com/nodejs/node/pull/52794/, https://github.com/nodejs/node/pull/52802/)
Appeared test-digitalocean-ubi81_container-x64-1, test-equinix-ubuntu2004_container-arm64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-6, test-equinix-rhel8_container-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58808/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58872/
Example ``` not ok 2515 parallel/test-net-write-fully-async-buffer --- duration_ms: 120092.79200 severity: fail exitcode: -15 stack: |- timeout ... ```

Reason parallel/test-crypto-keygen-non-standard-public-exponent
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52347/, https://github.com/nodejs/node/pull/52744/, https://github.com/nodejs/node/pull/52766/)
Appeared test-equinix-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2, test-equinix-debian11_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58801/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58876/
Example ``` not ok 3802 parallel/test-crypto-keygen-non-standard-public-exponent --- duration_ms: 360081.75900 severity: fail exitcode: -15 stack: |- timeout ... ```

Reason parallel/test-error-serdes
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52723/, https://github.com/nodejs/node/pull/52730/, https://github.com/nodejs/node/pull/52727/)
Appeared test-rackspace-win2022_vs2022-x64-3, test-equinix-ubuntu2004_container-arm64-2, test-digitalocean-ubuntu2204_sharedlibs_container-x64-6, test-ibm-ubuntu2204_sharedlibs_container-x64-3, test-azure_msft-win10_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/58781/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58788/
Example ``` not ok 942 parallel/test-error-serdes --- duration_ms: 120217.64000 severity: fail exitcode: 1 stack: |- timeout ... ```

Reason parallel/test-performance-function
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52747/, https://github.com/nodejs/node/pull/52763/, https://github.com/nodejs/node/pull/52768/)
Appeared undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/58796/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58832/
Example ``` not ok 2437 parallel/test-performance-function --- duration_ms: 409.53800 severity: fail exitcode: 1 stack: |- node:internal/event_target:1095 process.nextTick(() => { throw err; }); ^ RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be >= 1 && <= 9007199254740991. Received 0 at RecordableHistogram.record (node:internal/histogram:291:5) at processComplete (node:internal/perf/timerify:40:15) at timerified m (node:internal/perf/timerify:87:5) at /home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/parallel/test-performance-function.js:103:5 { code: 'ERR_OUT_OF_RANGE' } Node.js v22.1.0 ... ```

Reason parallel/test-runner-output
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52609/, https://github.com/nodejs/node/pull/52213/, https://github.com/nodejs/node/pull/52793/)
Appeared test-azure_msft-win10_vs2019-x64-1, test-azure_msft-win10_vs2019-x64-3, test-azure_msft-win11_vs2022-x64-1, test-rackspace-win2016_vs2015-x64-1, test-rackspace-win2019_vs2019-x64-3, test-rackspace-win2019_vs2019-x64-2, test-rackspace-win2022_vs2022-x64-3, test-rackspace-win2022_vs2022-x64-2, test-azure_msft-win11_vs2022-arm64-2, test-azure_msft-win11_vs2022-arm64-1, test-azure_msft-win2016_vs2017-x64-2, test-rackspace-win2022_vs2022-x64-6, test-azure_msft-win11_vs2022-arm64-5, test-osuosl-aix72-ppc64_be-3
First CI https://ci.nodejs.org/job/node-test-pull-request/58810/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58868/
Example ``` not ok 671 parallel/test-runner-output --- duration_ms: 5884.72800 severity: fail exitcode: 1 stack: |- TAP version 13 # Subtest: test runner output # Subtest: test-runner/output/abort.js ok 1 - test-runner/output/abort.js --- duration_ms: 2062.3982 ... ```

Reason parallel/test-cli-node-options
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52760/, https://github.com/nodejs/node/pull/52793/)
Appeared test-rackspace-win2022_vs2022-x64-4, test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58853/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58868/
Example ``` not ok 122 parallel/test-cli-node-options --- duration_ms: 1573.02300 severity: crashed exitcode: 2147483651 stack: |- Can't clean tmpdir: C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.124 Files blocking: [ 'node_trace.1.log' ] node:internal/child_process:421 throw new ErrnoException(err, 'spawn'); ^ Error: spawn UNKNOWN at ChildProcess.spawn (node:internal/child_process:421:11) at spawn (node:child_process:761:9) at execFile (node:child_process:351:17) at expect (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-cli-node-options.js:119:3) at Object. (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-cli-node-options.js:76:1) at Module._compile (node:internal/modules/cjs/loader:1358:14) at Module._extensions..js (node:internal/modules/cjs/loader:1416:10) at Module.load (node:internal/modules/cjs/loader:1208:32) at Mo... ```

Reason parallel/test-file-write-stream4
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52747/, https://github.com/nodejs/node/pull/52763/)
Appeared test-equinix-rhel8_container-arm64-1, test-orka-macos11-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58796/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58838/
Example ``` not ok 3875 parallel/test-file-write-stream4 --- duration_ms: 360075.81700 severity: fail exitcode: -15 stack: |- timeout ... ```

Reason parallel/test-macos-app-sandbox
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52730/, https://github.com/nodejs/node/pull/52047/)
Appeared test-orka-macos11-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/58784/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58792/
Example ``` not ok 1904 parallel/test-macos-app-sandbox --- duration_ms: 6418.62000 severity: fail exitcode: 1 stack: |- node:assert:126 throw new AssertionError(obj); ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 1 !== 0 at Object. (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/parallel/test-macos-app-sandbox.js:47:8) at Module._compile (node:internal/modules/cjs/loader:1480:14) at Module._extensions..js (node:internal/modules/cjs/loader:1564:10) at Module.load (node:internal/modules/cjs/loader:1287:32) at Module._load (node:internal/modules/cjs/loader:1103:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:168:12) at node:internal/main/run_main_module:30:49 { generatedMessage: true, code: 'ERR_ASSERTION', actual: 1, expected: 0, operator: 'strictEqual' } Node.js v23.0.0-pre ... ```

Reason parallel/test-performance-eventlooputil
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52627/, https://github.com/nodejs/node/pull/52609/)
Appeared undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/58802/
Last CI https://ci.nodejs.org/job/node-test-pull-request/58827/
Example ``` not ok 2434 parallel/test-performance-eventlooputil --- duration_ms: 341.81800 severity: fail exitcode: 1 stack: |- node:internal/event_target:1095 process.nextTick(() => { throw err; }); ^ AssertionError [ERR_ASSERTION]: 131.10266304016113 < 131.10266305859375 at Timeout.runCalcTest (/home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/parallel/test-performance-eventlooputil.js:85:10) at Timeout.runCalcTest [as _onTimeout] (/home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/common/index.js:469:15) at listOnTimeout (node:internal/timers:575:11) at process.processTimers (node:internal/timers:514:7) { generatedMessage: false, code: 'ERR_ASSERTION', actual: false, expected: true, operator: '==' } Node.js v23.0.0-pre ... ```

Progress