Fix flaky networking in GitHub Actions

[iainelder]

A selection of recent failures caused by what appears to be flaky networking.

GnuCash 3: test_idempotent_installation (ubuntu:20.04)

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/h/humanity-icon-theme/humanity-icon-theme_0.6.15_all.deb  Undetermined Error [IP: 185.125.190.36 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fetched 261 MB in 1min 42s (2563 kB/s)

diff-pdf: test_idempotent_installation (ubuntu:20.04)

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/cracklib2/cracklib-runtime_2.9.6-3.2_amd64.deb  Undetermined Error [IP: 185.125.190.36 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fetched 291 MB in 3min 19s (1464 kB/s)

NICE DCV: test_idempotent_installation (ubuntu:20.04)

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libe/libedit/libedit2_3.1-20191231-1_amd64.deb  Undetermined Error [IP: 185.125.190.36 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fetched 90.0 MB in 1min 43s (877 kB/s)

OpenRefine: test_idempotent_installation (ubuntu:20.04)

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/a/at-spi2-atk/libatk-bridge2.0-0_2.34.2-0ubuntu2~20.04.1_amd64.deb  Undetermined Error [IP: 185.125.190.36 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fetched 95.3 MB in 14s (6850 kB/s)

Spotify: test_idempotent_installation (ubuntu:22.04)

E: Failed to fetch http://repository.spotify.com/pool/non-free/s/spotify-client/spotify-client_1.2.31.1205.g4d59ad7c_amd64.deb  Error reading from server. Remote end closed connection [IP: 13.33.62.2 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fetched 173 MB in 28s (6298 kB/s)

PlantUML: test_idempotent_installation (ubuntu:20.04)

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/a/at-spi2-atk/libatk-bridge2.0-0_2.34.2-0ubuntu2~20.04.1_amd64.deb  Undetermined Error [IP: 185.125.190.36 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fetched 95.3 MB in 20s (4697 kB/s)

Spotify: test_idempotent_installation (ubuntu:22.04)

E: Failed to fetch http://repository.spotify.com/pool/non-free/s/spotify-client/spotify-client_1.2.31.1205.g4d59ad7c_amd64.deb  Error reading from server. Remote end closed connection [IP: 13.33.62.127 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Fetched 173 MB in 28s (6294 kB/s)

VirtualBox: test_idempotent_installation (ubuntu:22.04)

+ sudo gpg --dearmor --yes --output /usr/share/keyrings/oracle-virtualbox-2016.gpg
gpg: no valid OpenPGP data found.

Angle-Grinder: test_idempotent_installation (ubuntu:22.04)

+ curl -Ss https://api.github.com/repos/rcoh/angle-grinder/releases/latest
++ command curl --netrc -Ss https://api.github.com/repos/rcoh/angle-grinder/releases/latest
++ jq -r '.assets[] | select(.name == "agrind-x86_64-unknown-linux-musl.tar.gz") | .browser_download_url'
+ browser_download_url=https://github.com/rcoh/angle-grinder/releases/download/v0.19.2/agrind-x86_64-unknown-linux-musl.tar.gz
++ curl --silent --show-error --url https://github.com/rcoh/angle-grinder/releases/download/v0.19.2/agrind-x86_64-unknown-linux-musl.tar.gz --location --remote-name --write-out '%{filename_effective}'
++ command curl --netrc --silent --show-error --url https://github.com/rcoh/angle-grinder/releases/download/v0.19.2/agrind-x86_64-unknown-linux-musl.tar.gz --location --remote-name --write-out '%{filename_effective}'
curl: (6) Could not resolve host: objects.githubusercontent.com
+ download_filename=agrind-x86_64-unknown-linux-musl.tar.gz

[iainelder]

https://github.com/actions/runner-images/issues/1187 describes the general problem.

Situations that require networking, such as downloading a file using curl or running tests that utilise localhost, are proving to be very flaky, with spurious timeouts occurring often.

And a solution:

It seems that your problem can be tied with network offloading, that is turned on by default. Could you please try to disable TCP/UDP offload before build, it can be done literally with one-line: sudo ethtool -K eth0 tx off rx off Reloading/restarts are not required, performance should not be affected.

Which has been packaged into a GitHub action until GitHub resolves the upstream issue.

https://github.com/smorimoto/tune-github-hosted-runner-network

[iainelder]

I used the action to tune the network. Let's see whether that fixes it.

[iainelder]

Another approach: accept that the network is unreliable and configure the tools to retry on connection failures.

apt has the Acquire::Retries setting. See Ask Ubuntu and the apt.conf man page.

curl has options --retry, --retry-connrefused, and --retry-all-errors. See Stack Overflow and the curl manual.