search

dusty-nv / jetson-containers

Machine Learning Containers for NVIDIA Jetson and JetPack-L4T
MIT License
2.3k stars 474 forks source link

When Building Pytorch:1.12 I am getting GPG issues from build essentials base image #305

Closed dustycodes closed 1 year ago

dustycodes commented 1 year ago

Just running: ./build.sh pytorch:1.12 It worked last friday but now it seems like there are some repos that are down. No issues updating on my orin just in the containers. I'll try again later but I thought I would report an issue.

Here is my log:

docker build --network=host --tag pytorch:1.12-r35.2.1-build-essential \
--file /home/paladin/jetson-containers/packages/build-essential/Dockerfile \
--build-arg BASE_IMAGE=nvcr.io/nvidia/l4t-jetpack:r35.2.1 \
/home/paladin/jetson-containers/packages/build-essential \
2>&1 | tee /home/paladin/jetson-containers/logs/20231018_123903/build/pytorch_1.12-r35.2.1-build-essential.txt; exit ${PIPESTATUS[0]}

DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  12.29kB
Step 1/5 : ARG BASE_IMAGE
Step 2/5 : FROM ${BASE_IMAGE}
 ---> c36b76efa9e5
Step 3/5 : ENV DEBIAN_FRONTEND=noninteractive
 ---> Using cache
 ---> caaff82891e3
Step 4/5 : RUN apt-get update &&        apt-get install --fix-missing &&        apt-get install -y --no-install-recommends      build-essential         software-properties-common      apt-transport-https     ca-certificates         lsb-release     pkg-config      gnupg   git     wget    curl nano     zip     unzip   && rm -rf /var/lib/apt/lists/*  && apt-get clean
 ---> Running in ec09bf1c6dd2
Get:1 https://repo.download.nvidia.com/jetson/common r35.2 InRelease [2555 B]
Err:1 https://repo.download.nvidia.com/jetson/common r35.2 InRelease
  Couldn't create temporary file /tmp/apt.conf.WraApg for passing config to apt-key
Hit:2 http://ports.ubuntu.com/ubuntu-ports focal InRelease
Err:2 http://ports.ubuntu.com/ubuntu-ports focal InRelease
  Couldn't create temporary file /tmp/apt.conf.rCbpqj for passing config to apt-key
Get:3 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease [114 kB]
Err:3 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease
  Couldn't create temporary file /tmp/apt.conf.kxibZm for passing config to apt-key
Get:4 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease [108 kB]
Err:4 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease
  Couldn't create temporary file /tmp/apt.conf.gpQXBs for passing config to apt-key
Get:5 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease [114 kB]
Err:5 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease
  Couldn't create temporary file /tmp/apt.conf.VS2WNw for passing config to apt-key
Fetched 339 kB in 1s (250 kB/s)
Reading package lists...
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://repo.download.nvidia.com/jetson/common r35.2 InRelease: Couldn't create temporary file /tmp/apt.conf.WraApg for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://ports.ubuntu.com/ubuntu-ports focal InRelease: Couldn't create temporary file /tmp/apt.conf.rCbpqj for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease: Couldn't create temporary file /tmp/apt.conf.kxibZm for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease: Couldn't create temporary file /tmp/apt.conf.gpQXBs for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://ports.ubuntu.com/ubuntu-ports focal-security InRelease: Couldn't create temporary file /tmp/apt.conf.VS2WNw for passing config to apt-key
W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/focal/InRelease  Couldn't create temporary file /tmp/apt.conf.rCbpqj for passing config to apt-key
W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/focal-updates/InRelease  Couldn't create temporary file /tmp/apt.conf.kxibZm for passing config to apt-key
W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/focal-backports/InRelease  Couldn't create temporary file /tmp/apt.conf.gpQXBs for passing config to apt-key
W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/focal-security/InRelease  Couldn't create temporary file /tmp/apt.conf.VS2WNw for passing config to apt-key
W: Failed to fetch https://repo.download.nvidia.com/jetson/common/dists/r35.2/InRelease  Couldn't create temporary file /tmp/apt.conf.WraApg for passing config to apt-key
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists...
Building dependency tree...
Reading state information...
build-essential is already the newest version (12.8ubuntu1.1).
build-essential set to manually installed.
ca-certificates is already the newest version (20211016ubuntu0.20.04.1).
gnupg is already the newest version (2.2.19-3ubuntu2.2).
gnupg set to manually installed.
wget is already the newest version (1.20.3-1ubuntu2).
The following additional packages will be installed:
  distro-info-data gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 git-man
  libappstream4 libcurl4 liberror-perl libgirepository-1.0-1 libglib2.0-bin
  libglib2.0-data liblmdb0 libpackagekit-glib2-18 libpolkit-agent-1-0
  libpolkit-gobject-1-0 libstemmer0d libyaml-0-2 packagekit policykit-1
  python-apt-common python3-apt python3-certifi python3-chardet python3-dbus
  python3-gi python3-idna python3-requests python3-requests-unixsocket
  python3-six python3-software-properties python3-urllib3
Suggested packages:
  gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-el git-email
  git-gui gitk gitweb git-cvs git-mediawiki git-svn hunspell appstream
  python3-apt-dbg python-apt-doc python-dbus-doc python3-dbus-dbg
  python3-cryptography python3-openssl python3-socks
Recommended packages:
  less ssh-client packagekit-tools unattended-upgrades
The following NEW packages will be installed:
  apt-transport-https curl distro-info-data gir1.2-glib-2.0
  gir1.2-packagekitglib-1.0 git git-man libappstream4 libcurl4 liberror-perl
  libgirepository-1.0-1 libglib2.0-bin libglib2.0-data liblmdb0
  libpackagekit-glib2-18 libpolkit-agent-1-0 libpolkit-gobject-1-0
  libstemmer0d libyaml-0-2 lsb-release nano packagekit pkg-config policykit-1
  python-apt-common python3-apt python3-certifi python3-chardet python3-dbus
  python3-gi python3-idna python3-requests python3-requests-unixsocket
  python3-six python3-software-properties python3-urllib3
  software-properties-common unzip zip
0 upgraded, 39 newly installed, 0 to remove and 0 not upgraded.
Need to get 8325 kB of archives.
After this operation, 50.7 MB of additional disk space will be used.
Err:1 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 distro-info-data all 0.43ubuntu1.11
  404  Not Found [IP: 185.125.190.39 80]
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libgirepository-1.0-1 arm64 1.64.1-1~ubuntu20.04.1 [77.0 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 gir1.2-glib-2.0 arm64 1.64.1-1~ubuntu20.04.1 [134 kB]
Err:4 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libglib2.0-data all 2.64.6-1~ubuntu20.04.4
  404  Not Found [IP: 185.125.190.39 80]
Get:5 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 libyaml-0-2 arm64 0.2.2-1 [44.4 kB]
Get:6 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 lsb-release all 11.1.0ubuntu2 [10.6 kB]
Get:7 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-dbus arm64 1.2.16-1build1 [91.4 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-gi arm64 3.36.0-1 [159 kB]
Get:9 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-six all 1.14.0-2 [12.1 kB]
Get:10 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 liblmdb0 arm64 0.9.24-1 [43.1 kB]
Get:11 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libpolkit-gobject-1-0 arm64 0.105-26ubuntu1.3 [37.4 kB]
Get:12 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 nano arm64 4.8-1ubuntu1 [261 kB]
Err:13 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 python-apt-common all 2.0.1
  404  Not Found [IP: 185.125.190.39 80]
Err:14 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 python3-apt arm64 2.0.1
  404  Not Found [IP: 185.125.190.39 80]
Get:15 http://ports.ubuntu.com/ubuntu-ports focal-updates/universe arm64 apt-transport-https all 2.0.9 [1704 B]
Ign:16 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libcurl4 arm64 7.68.0-1ubuntu2.15
Ign:17 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 curl arm64 7.68.0-1ubuntu2.15
Get:18 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libpackagekit-glib2-18 arm64 1.1.13-2ubuntu1.1 [92.8 kB]
Get:19 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 gir1.2-packagekitglib-1.0 arm64 1.1.13-2ubuntu1.1 [21.6 kB]
Get:20 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 liberror-perl all 0.17029-1 [26.5 kB]
Ign:21 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 git-man all 1:2.25.1-1ubuntu3.8
Ign:22 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 git arm64 1:2.25.1-1ubuntu3.8
Get:23 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 libstemmer0d arm64 0+svn585-2 [60.6 kB]
Get:24 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 libappstream4 arm64 0.12.10-2 [120 kB]
Err:25 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libglib2.0-bin arm64 2.64.6-1~ubuntu20.04.4
  404  Not Found [IP: 185.125.190.39 80]
Get:26 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libpolkit-agent-1-0 arm64 0.105-26ubuntu1.3 [14.2 kB]
Get:27 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 pkg-config arm64 0.29.1-0ubuntu4 [43.8 kB]
Get:28 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 policykit-1 arm64 0.105-26ubuntu1.3 [79.4 kB]
Get:29 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-certifi all 2019.11.28-1 [149 kB]
Get:30 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-chardet all 3.0.4-4build1 [80.4 kB]
Get:31 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-idna all 2.8-1 [34.6 kB]
Get:32 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 python3-urllib3 all 1.25.8-2ubuntu0.2 [88.4 kB]
Get:33 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-requests all 2.22.0-2ubuntu1 [47.1 kB]
Get:34 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 python3-requests-unixsocket all 0.2.0-2 [7272 B]
Err:35 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 python3-software-properties all 0.99.9.10
  404  Not Found [IP: 185.125.190.39 80]
Get:36 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 packagekit arm64 1.1.13-2ubuntu1.1 [388 kB]
Err:37 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 software-properties-common all 0.99.9.10
  404  Not Found [IP: 185.125.190.39 80]
Get:38 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 unzip arm64 6.0-25ubuntu1.1 [162 kB]
Get:39 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 zip arm64 3.0-11build1 [146 kB]
Err:16 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 libcurl4 arm64 7.68.0-1ubuntu2.15
  404  Not Found [IP: 185.125.190.39 80]
Err:17 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 curl arm64 7.68.0-1ubuntu2.15
  404  Not Found [IP: 185.125.190.39 80]
Err:21 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 git-man all 1:2.25.1-1ubuntu3.8
  404  Not Found [IP: 185.125.190.39 80]
Err:22 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 git arm64 1:2.25.1-1ubuntu3.8
  404  Not Found [IP: 185.125.190.39 80]
Fetched 2433 kB in 5s (488 kB/s)
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/d/distro-info-data/distro-info-data_0.43ubuntu1.11_all.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/g/glib2.0/libglib2.0-data_2.64.6-1~ubuntu20.04.4_all.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/p/python-apt/python-apt-common_2.0.1_all.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/p/python-apt/python3-apt_2.0.1_arm64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/c/curl/libcurl4_7.68.0-1ubuntu2.15_arm64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/c/curl/curl_7.68.0-1ubuntu2.15_arm64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/g/git/git-man_2.25.1-1ubuntu3.8_all.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/g/git/git_2.25.1-1ubuntu3.8_arm64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/g/glib2.0/libglib2.0-bin_2.64.6-1~ubuntu20.04.4_arm64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/s/software-properties/python3-software-properties_0.99.9.10_all.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/s/software-properties/software-properties-common_0.99.9.10_all.deb  404  Not Found [IP: 185.125.190.39 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get update &&       apt-get install --fix-missing &&        apt-get install -y --no-install-recommends      build-essential         software-properties-common      apt-transport-https     ca-certificates         lsb-release     pkg-config      gnupg   git     wget curl     nano    zip     unzip   && rm -rf /var/lib/apt/lists/*  && apt-get clean' returned a non-zero code: 100
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/paladin/jetson-containers/jetson_containers/build.py", line 96, in <module>
    build_container(args.name, args.packages, args.base, args.build_flags, args.simulate, args.skip_tests, args.test_only, args.push, args.no_github_api)
  File "/home/paladin/jetson-containers/jetson_containers/container.py", line 141, in build_container
    status = subprocess.run(cmd.replace(_NEWLINE_, ' '), executable='/bin/bash', shell=True, check=True)  
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paladin/miniconda3/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'docker build --network=host --tag pytorch:1.12-r35.2.1-build-essential --file /home/paladin/jetson-containers/packages/build-essential/Dockerfile --build-arg BASE_IMAGE=nvcr.io/nvidia/l4t-jetpack:r35.2.1 /home/paladin/jetson-containers/packages/build-essential 2>&1 | tee /home/paladin/jetson-containers/logs/20231018_123903/build/pytorch_1.12-r35.2.1-build-essential.txt; exit ${PIPESTATUS[0]}' returned non-zero exit status 100.
dusty-nv commented 1 year ago

Hi @dustycodes, I just did a rebuild of this container with the command below, and it succeeded without issue:

./build.sh --build-flags='--no-cache' pytorch:1.12

It looks like there were some 404 Not Found [IP: 185.125.190.39 80] errors in your log - maybe it was a temporary networking issue somewhere?

dustycodes commented 1 year ago

That's what I was hoping. Thanks for verifying. I'll check again