FreeBSD builds timing out

[richardlau]

Starting 13 September the FreeBSD builds (Node.js 18 and 20 daily builds) are timing out in Jenkins.

https://ci.nodejs.org/job/node-test-commit-freebsd/nodes=freebsd12-x64/buildTimeTrend

e.g. https://ci.nodejs.org/job/node-test-commit-freebsd/nodes=freebsd12-x64/56720/console

12:39:48 ok 3786 async-hooks/test-zlib.zlib-binding.deflate
12:39:48   ---
12:39:48   duration_ms: 103.84000
12:39:48   ...
12:51:48 Build timed out (after 12 minutes). Marking the build as failed.
12:51:48 Build was aborted

[richardlau]

I extended the timeout from 12 mins to 30 mins but today's Node.js 18 daily build still timed out:

https://ci.nodejs.org/job/node-test-commit-freebsd/56843/nodes=freebsd12-x64/console

10:25:12 ok 3596 es-module/test-wasm-web-api
10:25:12   ---
10:25:12   duration_ms: 5397.05600
10:25:12   ...
10:55:12 Build timed out (after 30 minutes). Marking the build as failed.
10:55:12 Build was aborted

Since the failures are all on test-digitalocean-freebsd12-x64-2 -- I've temporarily forced a build onto the other machine (test-digitalocean-freebsd12-x64-1) to see if it's machine specific: https://ci.nodejs.org/job/node-test-commit-freebsd/56845/nodes=freebsd12-x64/console

[richardlau]

Ansible playbook no longer works on FreeBSD 12 as that is End-of-Life:

TASK [package-upgrade : upgrade installed packages] *************************************************************************************************************************************************************
fatal: [test-digitalocean-freebsd12-x64-2]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 3, "stderr": "Shared connection to 107.170.28.213 closed.\r\n", "stderr_lines": ["Shared connection to 107.170.28.213 closed."], "stdout": "pkg: http://pkgmir.geo.freebsd.org/FreeBSD:12:amd64/latest/meta.txz: Not Found\r\npkg: http://pkgmir.geo.freebsd.org/FreeBSD:12:amd64/latest/packagesite.pkg: Not Found\r\npkg: http://pkgmir.geo.freebsd.org/FreeBSD:12:amd64/latest/packagesite.txz: Not Found\r\n", "stdout_lines": ["pkg: http://pkgmir.geo.freebsd.org/FreeBSD:12:amd64/latest/meta.txz: Not Found", "pkg: http://pkgmir.geo.freebsd.org/FreeBSD:12:amd64/latest/packagesite.pkg: Not Found", "pkg: http://pkgmir.geo.freebsd.org/FreeBSD:12:amd64/latest/packagesite.txz: Not Found"]}

PLAY RECAP ******************************************************************************************************************************************************************************************************
test-digitalocean-freebsd12-x64-2 : ok=7    changed=1    unreachable=0    failed=1    skipped=3    rescued=0    ignored=0

I've rebooted the test-digitalocean-freebsd12-x64-2 and started another build: https://ci.nodejs.org/job/node-test-commit-freebsd/56851/nodes=freebsd12-x64/

[richardlau]

I've rebooted the test-digitalocean-freebsd12-x64-2 and started another build: https://ci.nodejs.org/job/node-test-commit-freebsd/56851/nodes=freebsd12-x64/

That one passed. Optimistically closing.