bazelbuild / bazel-buildfarm

Bazel remote caching and execution service
https://bazel.build
Apache License 2.0
635 stars 199 forks source link

[Bazel CI] apt-get and integration tests are failing with Bazel@HEAD #1759

Closed sgowroji closed 4 weeks ago

sgowroji commented 4 weeks ago

CI: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3907#018ff0e0-6b68-4847-b075-51ae81dc16db

Platform: Ubuntu

Logs:


200.8 Fetched 166 MB in 3min 19s (833 kB/s)
--
  | 200.8 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/g/gtk+2.0/libgail-common_2.24.32-1ubuntu1_amd64.deb  Undetermined Error [IP: 185.125.190.36 80]
  | 200.8 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  | ------
  | Dockerfile:10
  | --------------------
  | 8 \|
  | 9 \|     RUN apt-get update
  | 10 \| >>> RUN apt-get -y install wget git zip python gcc openjdk-8-jdk g++ redis redis-server
  | 11 \|     RUN wget --tries=10 -O get-pip.py https://bootstrap.pypa.io/pip/2.7/get-pip.py
  | 12 \|     RUN python2 get-pip.py
  | --------------------
  | ERROR: failed to solve: process "/bin/sh -c apt-get -y install wget git zip python gcc openjdk-8-jdk g++ redis redis-server" did not complete successfully: exit code: 100
  | Unable to find image 'buildfarm:latest' locally
  | docker: Error response from daemon: pull access denied for buildfarm, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
  | See 'docker run --help'.
  | buildfarm-redis
  | Traceback (most recent call last):
  | File "bazelci.py", line 4468, in <module>
  | sys.exit(main())
  | File "bazelci.py", line 4436, in main
  | execute_commands(
  | File "bazelci.py", line 1335, in execute_commands
  | PrepareRepoInCwd(True, initial_setup=True)
  | File "bazelci.py", line 1333, in PrepareRepoInCwd
  | execute_shell_commands(task_config.get("shell_commands", None), print_cmd_groups)
  | File "bazelci.py", line 1852, in execute_shell_commands
  | execute_command([shell_command], shell=True)
  | File "bazelci.py", line 2716, in execute_command
  | return subprocess.run(
  | File "/usr/lib/python3.8/subprocess.py", line 516, in run
  | raise CalledProcessError(retcode, process.args,
  | subprocess.CalledProcessError: Command '['set -e\nexport RUN_TEST=./.bazelci/run_abseil_test.sh\nexport BUILDFARM_CONFIG="/buildfarm/examples/config.minimal.yml"\nexport EXECUTION_STAGE_WIDTH=10\nexport TEST_ARG1=--experimental_remote_cache_compression\n./.bazelci/integration_test.sh']' returned non-zero exit status 125.

CC Greenteam @wyverald

werkt commented 4 weeks ago

Transient issue with ubuntu upstream:

200.8 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/g/gtk+2.0/libgail-common_2.24.32-1ubuntu1_amd64.deb  Undetermined Error [IP: 185.125.190.36 80]

Not sure if there's a mirror that's more reliable for the build hosts that might be better to point to...