Tracking issue: Networking Issues

adamfarley commented 1 year ago

Summary

This issue is for storing details of networking issues seen during triage or otherwise.

Details

Each entry should include the date, host, error message, and a URL.

The problems listed here can have issues elsewhere, but are primarily for unpredictable, temporary issues.

Examples:

Lost connection between Jenkins and host.
Lost connection between host and github.com.
Corrupted downloads from a server somewhere (such as Maven) to the host.

sxa commented 1 year ago

GitHub access issue on [mac installer creation](job https://ci.adoptium.net/job/build-scripts/job/release/job/create_installer_mac/10953/console) (No machine selected at this point)

ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://github.com/adoptium/installer.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: unable to access 'https://github.com/adoptium/installer.git/': Failed to connect to github.com port 443: Operation timed out

adamfarley commented 11 months ago

https://ci.adoptium.net/job/Test_openjdk17_hs_sanity.openjdk_aarch64_linux/373/consoleFull - test-docker-centos8-armv8-1 https://ci.adoptium.net/job/Test_openjdk17_hs_extended.openjdk_aarch64_linux_testList_1/42/console - test-docker-ubuntu1804-armv8l-4

Exception: org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for test-docker-ubuntu1804-armv8l-4; test-docker-ubuntu1804-armv8l-4 was marked offline: Connection was broken

The failures happened about an hour apart, but the wording is about the same.

adamfarley commented 11 months ago

Two tests failed because they couldn't download renaissance.jar for performance runs, as part of (or just after) the liberty setup.

URL seems fine on my machine, so assuming it's a temporary upstream networking/server issue until future failures prove otherwise.

https://ci.adoptium.net/job/Test_openjdk11_hs_sanity.perf_x86-64_linux/884/console https://ci.adoptium.net/job/Test_openjdk11_hs_extended.perf_x86-64_linux/161/console

Both ran on test-equinix_esxi-ubuntu2204-x64-1

adamfarley commented 11 months ago

https://ci.adoptium.net/job/Test_openjdk21_hs_extended.openjdk_x86-64_mac_testList_1/31/console

Cannot contact test-orka-macos14-x64-96vzx: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel

smlambert commented 11 months ago

https://github.com/adoptium/infrastructure/issues/3190#issuecomment-1841094904 - should be resolved by https://github.com/adoptium/aqa-tests/pull/4903 once merged.

sxa commented 11 months ago

Two tests failed because they couldn't download renaissance.jar for performance runs, as part of (or just after) the liberty setup. URL seems fine on my machine, so assuming it's a temporary upstream networking/server issue until future failures prove otherwise.

Those are both disk space issues, not networking ones.

18:03:21  tee: /home/jenkins/workspace/Test_openjdk11_hs_sanity.perf_x86-64_linux/aqa-tests/TKG/../TKG/output_compilation/compilation.log: No space left on device

18:07:00      [retry] Attempt [2]: error occurred; retrying...tee: /home/jenkins/workspace/Test_openjdk11_hs_extended.perf_x86-64_linux/aqa-tests/TKG/../TKG/output_compilation/compilation.log: No space left on device

adamfarley commented 11 months ago

As noted here, JDK8u is missing a large number of published binaries from the temurin8-binaries repo.

I think that is because the jdk8 build pipeline on the 11th timed out while uploading the binaries for some reason. Not sure if this was because of a hang or just slow uploads, as the upload job seems to lack regular time stamps.

Currently my plan is to ignore this unless it becomes a pattern.