search

NVIDIA / nvidia-docker

Build and run Docker containers leveraging NVIDIA GPUs
Apache License 2.0
17.17k stars 2.03k forks source link

Can't download cuda on Ubuntu 20.04 #1778

Closed ukby1234 closed 10 months ago

ukby1234 commented 10 months ago

1. Issue or feature description

It fails with 

Get:1 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  InRelease [1581 B]
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]                   
Get:3 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]                        
Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages [1127 kB]
Err:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages
  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.195.19.142 443]
  Hashes of expected file:
   - Filesize:1126689 [weak]
   - SHA256:7cee2584ca6d97b2f07018ba4f9c3c473fb4e299ed170968a1f8c99c090cc59f
   - SHA1:4ee24fac5518a3fcc3702590a0dab32c95484c54 [weak]
   - MD5Sum:593faff511765d11055c9919bf2e3bf8 [weak]
  Release file created at: Thu, 17 Aug 2023 19:03:05 +0000
Get:7 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [29.3 kB]
Get:8 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1092 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2648 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]               
Get:13 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2938 kB]      
Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]             
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3430 kB]          
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [32.0 kB]                                                                                                                                                           
Get:17 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2795 kB]                                                                                                                                                           
Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1394 kB]                                                                                                                                                             
Get:19 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [55.2 kB]                                                                                                                                                               
Get:20 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [28.6 kB]

Looks like the repository is in a bad state.

2. Steps to reproduce the issue

docker run --platform linux/amd64 -it nvidia/cuda:12.2.0-devel-ubuntu20.04 bash
apt-get update

3. Information to attach (optional if deemed irrelevant)

Askannz commented 10 months ago

Same here. None of the docker images based on Ubuntu 20.04 seem to work.

dazzleckc commented 10 months ago

Facing the same issue. All Docker images built on Ubuntu 20.04 are encountering this problem.

FROM  nvidia/cuda:12.1.0-runtime-ubuntu20.04
RUN apt-get update -y

RUN apt-get update -y:                                                                                                                                                                                                               
1.596 Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]                                                                                                                                                               
2.036 Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]                                                                                                                                                                         
2.796 Get:3 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [29.3 kB]                                                                                                                                              
3.331 Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]                                                                                                                                                                 
3.525 Get:5 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64  InRelease [1581 B]
3.654 Get:6 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64  Packages [1127 kB]
3.655 Ign:6 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64  Packages
4.507 Get:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
5.782 Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2648 kB]
8.310 Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2938 kB]
10.72 Get:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1092 kB]
10.84 Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
11.72 Ign:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages
11.80 Get:12 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
12.05 Err:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages
12.05   File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 180.119.146.98 443]
13.61 Get:13 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
15.97 Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
16.79 Get:15 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [32.0 kB]
19.06 Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1393 kB]
22.71 Get:17 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2795 kB]
25.90 Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3430 kB]
26.82 Get:19 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [28.6 kB]
28.08 Get:20 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [55.2 kB]
28.10 Fetched 27.9 MB in 28s (1000 kB/s)
28.10 Reading package lists...
28.99 E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/Packages  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 180.119.146.98 443]
28.99 E: Some index files failed to download. They have been ignored, or old ones used instead.
------
dockerfile:8
--------------------
   6 |     
   7 |     RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
   8 | >>> RUN apt-get update -y
   9 |     RUN apt-get install python3 python3-pip curl libgl1 libglib2.0-0 -y
  10 |     RUN apt-get clean
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update -y" did not complete successfully: exit code: 100
tbcashy commented 10 months ago

Same issue here with Ubuntu 20.04:

ubuntu@ip-10-100-2-74:~$ sudo apt update
Hit:1 http://eu-central-1.ec2.archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://eu-central-1.ec2.archive.ubuntu.com/ubuntu focal-updates InRelease                                                                                        
Hit:3 http://eu-central-1.ec2.archive.ubuntu.com/ubuntu focal-backports InRelease                                                                                      
Hit:4 https://download.docker.com/linux/ubuntu focal InRelease                                                                                                         
Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease                                                                                                       
Get:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  InRelease [1581 B]
Get:7 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages [1127 kB]
Err:7 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages
  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.199.20.126 443]
  Hashes of expected file:
   - Filesize:1126689 [weak]
   - SHA256:7cee2584ca6d97b2f07018ba4f9c3c473fb4e299ed170968a1f8c99c090cc59f
   - SHA1:4ee24fac5518a3fcc3702590a0dab32c95484c54 [weak]
   - MD5Sum:593faff511765d11055c9919bf2e3bf8 [weak]
  Release file created at: Thu, 17 Aug 2023 19:03:05 +0000
Fetched 1581 B in 0s (3737 B/s)
Reading package lists... Done
E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/Packages.gz  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.199.20.126 443]
   Hashes of expected file:
    - Filesize:1126689 [weak]
    - SHA256:7cee2584ca6d97b2f07018ba4f9c3c473fb4e299ed170968a1f8c99c090cc59f
    - SHA1:4ee24fac5518a3fcc3702590a0dab32c95484c54 [weak]
    - MD5Sum:593faff511765d11055c9919bf2e3bf8 [weak]
   Release file created at: Thu, 17 Aug 2023 19:03:05 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.
estan commented 10 months ago

It's https://github.com/NVIDIA/nvidia-docker/issues/1633 again.

NVIDIA: Your file system should support atomic moves. Use it.

starburst997 commented 10 months ago

Temp fix: rm /etc/apt/sources.list.d/cuda.list

gsaluja9 commented 10 months ago

Same in our CI pipelines . Is there any ETA for a fix?

Askannz commented 10 months ago

Seems fixed now.

tbcashy commented 10 months ago

@Askannz Yes, can confirm!