search

NVIDIA / nvidia-docker

Build and run Docker containers leveraging NVIDIA GPUs
Apache License 2.0
17.17k stars 2.03k forks source link

Unable to find image 'nvidia/cuda:11.0-base' locally when testing nvidia-docker2 setup #1794

Closed monajalal closed 7 months ago

monajalal commented 7 months ago

I am following the official NVIDIA instructions for installing nvidia-docker2. I get this error.

(base) mona@ada:~/clean-pvnet/docker$ curl https://get.docker.com | sh \
  && sudo systemctl --now enable docker
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21927  100 21927    0     0   186k      0 --:--:-- --:--:-- --:--:--  187k
# Executing docker install script, commit: e5543d473431b782227f8908005543bb4389b8de
Warning: the "docker" command appears to already exist on this system.

If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.

If you installed the current Docker package using this script and are using it
again to update Docker, you can safely ignore this message.

You may press Ctrl+C now to abort this script.
+ sleep 20
^C
(base) mona@ada:~/clean-pvnet/docker$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK
deb https://nvidia.github.io/libnvidia-container/stable/ubuntu18.04/$(ARCH) /
#deb https://nvidia.github.io/libnvidia-container/experimental/ubuntu18.04/$(ARCH) /
deb https://nvidia.github.io/nvidia-container-runtime/stable/ubuntu18.04/$(ARCH) /
#deb https://nvidia.github.io/nvidia-container-runtime/experimental/ubuntu18.04/$(ARCH) /
deb https://nvidia.github.io/nvidia-docker/ubuntu18.04/$(ARCH) /
(base) mona@ada:~/clean-pvnet/docker$ curl -s -L https://nvidia.github.io/nvidia-container-runtime/experimental/$distribution/nvidia-container-runtime.list | sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list
deb https://nvidia.github.io/libnvidia-container/experimental/ubuntu18.04/$(ARCH) /
deb https://nvidia.github.io/nvidia-container-runtime/experimental/ubuntu18.04/$(ARCH) /
(base) mona@ada:~/clean-pvnet/docker$ sudo apt-get update
Get:1 file:/var/cudnn-local-repo-ubuntu2204-8.9.5.29  InRelease [1,572 B]
Ign:2 http://10.82.164.106/debs ./ InRelease
Get:1 file:/var/cudnn-local-repo-ubuntu2204-8.9.5.29  InRelease [1,572 B]                               
Ign:3 http://10.82.164.106/debs ./ Release                                                                                           
Ign:4 http://10.82.164.106/debs ./ Packages                                                                                   
Ign:5 http://10.82.164.106/debs ./ Translation-en_US                                                   
Ign:6 http://10.82.164.106/debs ./ Translation-en                                                                            
Ign:4 http://10.82.164.106/debs ./ Packages                                                                                  
Ign:5 http://10.82.164.106/debs ./ Translation-en_US                                                   
Ign:6 http://10.82.164.106/debs ./ Translation-en                                                      
Ign:4 http://10.82.164.106/debs ./ Packages                         
Ign:5 http://10.82.164.106/debs ./ Translation-en_US                
Ign:6 http://10.82.164.106/debs ./ Translation-en                   
Hit:4 http://10.82.164.106/debs ./ Packages                         
Ign:5 http://10.82.164.106/debs ./ Translation-en_US                
Get:7 https://nvidia.github.io/libnvidia-container/experimental/ubuntu18.04/amd64  InRelease [1,503 B]
Ign:6 http://10.82.164.106/debs ./ Translation-en                                              
Get:8 https://nvidia.github.io/nvidia-container-runtime/experimental/ubuntu18.04/amd64  InRelease [1,494 B]
Ign:5 http://10.82.164.106/debs ./ Translation-en_US                             
Hit:9 http://us.archive.ubuntu.com/ubuntu jammy InRelease
Ign:6 http://10.82.164.106/debs ./ Translation-en                   
Ign:5 http://10.82.164.106/debs ./ Translation-en_US                
Get:10 https://nvidia.github.io/libnvidia-container/stable/ubuntu18.04/amd64  InRelease [1,484 B]
Hit:11 http://us.archive.ubuntu.com/ubuntu jammy-updates InRelease                                                                                                                                         
Ign:6 http://10.82.164.106/debs ./ Translation-en                                                                                                                                                          
Ign:5 http://10.82.164.106/debs ./ Translation-en_US                                                                                                                                                       
Ign:6 http://10.82.164.106/debs ./ Translation-en                                                                                                                                                          
Hit:12 https://nvidia.github.io/nvidia-container-runtime/stable/ubuntu18.04/amd64  InRelease                                                                                                               
Get:13 https://nvidia.github.io/nvidia-docker/ubuntu18.04/amd64  InRelease [1,474 B]                                                                                                                       
Hit:14 http://us.archive.ubuntu.com/ubuntu jammy-backports InRelease                                                                                                                                       
Hit:15 http://security.ubuntu.com/ubuntu jammy-security InRelease                                                                                                                                          
Hit:16 https://dl.google.com/linux/chrome/deb stable InRelease                                                                                                                                             
Hit:17 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  InRelease                                                                                                               
Hit:18 http://packages.ros.org/ros2/ubuntu jammy InRelease                                                                                                                                                 
Hit:19 https://packages.microsoft.com/repos/azure-cli jammy InRelease                                                                                                                                      
Hit:20 https://packages.microsoft.com/repos/ms-teams stable InRelease                                                                                        
Hit:21 https://packages.microsoft.com/repos/code stable InRelease                                                                      
Get:22 https://nvidia.github.io/libnvidia-container/experimental/ubuntu18.04/amd64  Packages [12.4 kB]                                
Get:23 https://nvidia.github.io/nvidia-container-runtime/experimental/ubuntu18.04/amd64  Packages [976 B]                                                               
Hit:24 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease                                                     
Hit:25 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease                
Get:26 https://nvidia.github.io/nvidia-docker/ubuntu18.04/amd64  Packages [4,488 B]                
Hit:27 https://download.docker.com/linux/ubuntu jammy InRelease                                                 
Hit:28 https://librealsense.intel.com/Debian/apt-repo jammy InRelease        
Fetched 23.8 kB in 6s (4,201 B/s)
Reading package lists... Done
W: https://nvidia.github.io/libnvidia-container/experimental/ubuntu18.04/amd64/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://nvidia.github.io/nvidia-container-runtime/experimental/ubuntu18.04/amd64/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://nvidia.github.io/libnvidia-container/stable/ubuntu18.04/amd64/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://nvidia.github.io/nvidia-container-runtime/stable/ubuntu18.04/amd64/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://nvidia.github.io/nvidia-docker/ubuntu18.04/amd64/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
(base) mona@ada:~/clean-pvnet/docker$ sudo apt-get install -y nvidia-docker2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
nvidia-docker2 is already the newest version (2.13.0-1).
The following packages were automatically installed and are no longer required:
  cuda-cccl-11-7 cuda-command-line-tools-11-7 cuda-compiler-11-7 cuda-cudart-11-7 cuda-cudart-dev-11-7 cuda-cuobjdump-11-7 cuda-cupti-11-7 cuda-cupti-dev-11-7 cuda-cuxxfilt-11-7 cuda-demo-suite-11-7
  cuda-documentation-11-7 cuda-driver-dev-11-7 cuda-gdb-11-7 cuda-libraries-11-7 cuda-libraries-dev-11-7 cuda-memcheck-11-7 cuda-nsight-11-7 cuda-nsight-compute-11-7 cuda-nsight-systems-11-7
  cuda-nvcc-11-7 cuda-nvdisasm-11-7 cuda-nvml-dev-11-7 cuda-nvprof-11-7 cuda-nvprune-11-7 cuda-nvrtc-11-7 cuda-nvrtc-dev-11-7 cuda-nvtx-11-7 cuda-nvvp-11-7 cuda-runtime-11-7 cuda-sanitizer-11-7
  cuda-toolkit-11-7 cuda-toolkit-11-7-config-common cuda-tools-11-7 cuda-visual-tools-11-7 gds-tools-11-7 libcublas-11-7 libcublas-dev-11-7 libcufft-11-7 libcufft-dev-11-7 libcufile-11-7
  libcufile-dev-11-7 libcurand-11-7 libcurand-dev-11-7 libcusolver-11-7 libcusolver-dev-11-7 libcusparse-11-7 libcusparse-dev-11-7 libnpp-11-7 libnpp-dev-11-7 libnvidia-egl-wayland1 libnvjpeg-11-7
  libnvjpeg-dev-11-7 nsight-compute-2022.2.1 nsight-systems-2022.1.3
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 58 not upgraded.
(base) mona@ada:~/clean-pvnet/docker$ sudo systemctl restart docker
(base) mona@ada:~/clean-pvnet/docker$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
Unable to find image 'nvidia/cuda:11.0-base' locally
docker: Error response from daemon: manifest for nvidia/cuda:11.0-base not found: manifest unknown: manifest unknown.
See 'docker run --help'.

sys info:

(base) mona@ada:~$ nvidia-docker --version
Docker version 24.0.6, build ed223bc
(base) mona@ada:~$ docker --version
Docker version 24.0.6, build ed223bc
(base) mona@ada:~$ uname -a
Linux ada 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
(base) mona@ada:~$ lsb_release -a
LSB Version:    core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy

In case needed, I need nvidia-docker2 for running the commands in this tutorial. https://github.com/zju3dv/pvnet/blob/master/docker/how-to-docker.md 

(base) mona@ada:~$ sudo apt show nvidia-docker2
Package: nvidia-docker2
Version: 2.13.0-1
Priority: optional
Section: utils
Maintainer: NVIDIA CORPORATION <cudatools@nvidia.com>
Installed-Size: 27.6 kB
Depends: nvidia-container-toolkit (>= 1.13.0-1), docker-ce (>= 18.06.0~ce~3-0~ubuntu) | docker-ee (>= 18.06.0~ce~3-0~ubuntu) | docker.io (>= 18.06.0) | moby-engine
Breaks: nvidia-docker (<< 2.0.0)
Replaces: nvidia-docker (<< 2.0.0)
Homepage: https://github.com/NVIDIA/nvidia-docker/wiki
Download-Size: 6,876 B
APT-Manual-Installed: yes
APT-Sources: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
Description: nvidia-docker CLI wrapper
 Replaces nvidia-docker with a new implementation based on the NVIDIA Container Toolkit

N: There are 33 additional records. Please use the '-a' switch to see them.

The template below is mostly useful for bug reports and support questions. Feel free to remove anything which doesn't apply to you and add more information where it makes sense.

Also, before reporting a new issue, please make sure that:

1. Issue or feature description

2. Steps to reproduce the issue

3. Information to attach (optional if deemed irrelevant)

-- WARNING, the following logs are for debugging purposes only --

I1115 20:59:04.793989 44149 nvc.c:376] initializing library context (version=1.14.3, build=1eb5a30a6ad0415550a9df632ac8832bf7e2bbba) I1115 20:59:04.794032 44149 nvc.c:350] using root / I1115 20:59:04.794035 44149 nvc.c:351] using ldcache /etc/ld.so.cache I1115 20:59:04.794046 44149 nvc.c:352] using unprivileged user 1002:1002 I1115 20:59:04.794073 44149 nvc.c:393] attempting to load dxcore to see if we are running under Windows Subsystem for Linux (WSL) I1115 20:59:04.794276 44149 nvc.c:395] dxcore initialization failed, continuing assuming a non-WSL environment W1115 20:59:04.798538 44150 nvc.c:273] failed to set inheritable capabilities W1115 20:59:04.798638 44150 nvc.c:274] skipping kernel modules load due to failure I1115 20:59:04.799422 44151 rpc.c:71] starting driver rpc service I1115 20:59:04.809217 44152 rpc.c:71] starting nvcgo rpc service I1115 20:59:04.811583 44149 nvc_info.c:798] requesting driver information with '' I1115 20:59:04.813426 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvoptix.so.535.104.12 I1115 20:59:04.813511 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-tls.so.535.104.12 I1115 20:59:04.813546 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.535.104.12 I1115 20:59:04.813586 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.535.104.12 I1115 20:59:04.813627 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-pkcs11.so.535.104.12 I1115 20:59:04.813650 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-pkcs11-openssl3.so.535.104.12 I1115 20:59:04.813682 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.535.104.12 I1115 20:59:04.813723 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.535.104.12 I1115 20:59:04.813761 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-nvvm.so.535.104.12 I1115 20:59:04.813805 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-ngx.so.535.104.12 I1115 20:59:04.813840 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.535.104.12 I1115 20:59:04.813889 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.535.104.12 I1115 20:59:04.813925 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.535.104.12 I1115 20:59:04.813957 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.535.104.12 I1115 20:59:04.813989 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-fbc.so.535.104.12 I1115 20:59:04.814036 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.535.104.12 I1115 20:59:04.814080 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.104.12 I1115 20:59:04.814120 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-cfg.so.535.104.12 I1115 20:59:04.814161 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvidia-allocator.so.535.104.12 I1115 20:59:04.814206 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libnvcuvid.so.535.104.12 I1115 20:59:04.814481 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libcudadebugger.so.535.104.12 I1115 20:59:04.814510 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libcuda.so.535.104.12 I1115 20:59:04.814682 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.535.104.12 I1115 20:59:04.814713 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.535.104.12 I1115 20:59:04.814747 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.535.104.12 I1115 20:59:04.814781 44149 nvc_info.c:176] selecting /usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.535.104.12 I1115 20:59:04.814835 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-tls.so.535.104.12 I1115 20:59:04.814866 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-ptxjitcompiler.so.535.104.12 I1115 20:59:04.814911 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-opticalflow.so.535.104.12 I1115 20:59:04.814953 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-opencl.so.535.104.12 I1115 20:59:04.814986 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-ml.so.535.104.12 I1115 20:59:04.815026 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-glvkspirv.so.535.104.12 I1115 20:59:04.815058 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-glsi.so.535.104.12 I1115 20:59:04.815085 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-glcore.so.535.104.12 I1115 20:59:04.815116 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-fbc.so.535.104.12 I1115 20:59:04.815157 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-encode.so.535.104.12 I1115 20:59:04.815198 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvidia-eglcore.so.535.104.12 I1115 20:59:04.815227 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libnvcuvid.so.535.104.12 I1115 20:59:04.815281 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libcuda.so.535.104.12 I1115 20:59:04.815326 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libGLX_nvidia.so.535.104.12 I1115 20:59:04.815357 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libGLESv2_nvidia.so.535.104.12 I1115 20:59:04.815390 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libGLESv1_CM_nvidia.so.535.104.12 I1115 20:59:04.815419 44149 nvc_info.c:176] selecting /usr/lib/i386-linux-gnu/libEGL_nvidia.so.535.104.12 W1115 20:59:04.815441 44149 nvc_info.c:402] missing library libnvidia-nscq.so W1115 20:59:04.815449 44149 nvc_info.c:402] missing library libnvidia-gpucomp.so W1115 20:59:04.815451 44149 nvc_info.c:402] missing library libnvidia-fatbinaryloader.so W1115 20:59:04.815457 44149 nvc_info.c:402] missing library libnvidia-compiler.so W1115 20:59:04.815468 44149 nvc_info.c:402] missing library libvdpau_nvidia.so W1115 20:59:04.815470 44149 nvc_info.c:402] missing library libnvidia-ifr.so W1115 20:59:04.815473 44149 nvc_info.c:402] missing library libnvidia-cbl.so W1115 20:59:04.815478 44149 nvc_info.c:406] missing compat32 library libnvidia-cfg.so W1115 20:59:04.815484 44149 nvc_info.c:406] missing compat32 library libnvidia-nscq.so W1115 20:59:04.815488 44149 nvc_info.c:406] missing compat32 library libcudadebugger.so W1115 20:59:04.815494 44149 nvc_info.c:406] missing compat32 library libnvidia-gpucomp.so W1115 20:59:04.815501 44149 nvc_info.c:406] missing compat32 library libnvidia-fatbinaryloader.so W1115 20:59:04.815504 44149 nvc_info.c:406] missing compat32 library libnvidia-allocator.so W1115 20:59:04.815509 44149 nvc_info.c:406] missing compat32 library libnvidia-compiler.so W1115 20:59:04.815517 44149 nvc_info.c:406] missing compat32 library libnvidia-pkcs11.so W1115 20:59:04.815519 44149 nvc_info.c:406] missing compat32 library libnvidia-pkcs11-openssl3.so W1115 20:59:04.815530 44149 nvc_info.c:406] missing compat32 library libnvidia-nvvm.so W1115 20:59:04.815534 44149 nvc_info.c:406] missing compat32 library libnvidia-ngx.so W1115 20:59:04.815538 44149 nvc_info.c:406] missing compat32 library libvdpau_nvidia.so W1115 20:59:04.815543 44149 nvc_info.c:406] missing compat32 library libnvidia-ifr.so W1115 20:59:04.815545 44149 nvc_info.c:406] missing compat32 library libnvidia-rtcore.so W1115 20:59:04.815553 44149 nvc_info.c:406] missing compat32 library libnvoptix.so W1115 20:59:04.815559 44149 nvc_info.c:406] missing compat32 library libnvidia-cbl.so I1115 20:59:04.816377 44149 nvc_info.c:302] selecting /usr/bin/nvidia-smi I1115 20:59:04.816394 44149 nvc_info.c:302] selecting /usr/bin/nvidia-debugdump I1115 20:59:04.816411 44149 nvc_info.c:302] selecting /usr/bin/nvidia-persistenced I1115 20:59:04.816433 44149 nvc_info.c:302] selecting /usr/bin/nvidia-cuda-mps-control I1115 20:59:04.816448 44149 nvc_info.c:302] selecting /usr/bin/nvidia-cuda-mps-server W1115 20:59:04.816570 44149 nvc_info.c:428] missing binary nv-fabricmanager I1115 20:59:04.816622 44149 nvc_info.c:488] listing firmware path /lib/firmware/nvidia/535.104.12/gsp_ga10x.bin I1115 20:59:04.816629 44149 nvc_info.c:488] listing firmware path /lib/firmware/nvidia/535.104.12/gsp_tu10x.bin I1115 20:59:04.816646 44149 nvc_info.c:561] listing device /dev/nvidiactl I1115 20:59:04.816651 44149 nvc_info.c:561] listing device /dev/nvidia-uvm I1115 20:59:04.816654 44149 nvc_info.c:561] listing device /dev/nvidia-uvm-tools I1115 20:59:04.816659 44149 nvc_info.c:561] listing device /dev/nvidia-modeset I1115 20:59:04.816678 44149 nvc_info.c:346] listing ipc path /run/nvidia-persistenced/socket W1115 20:59:04.816695 44149 nvc_info.c:352] missing ipc path /var/run/nvidia-fabricmanager/socket W1115 20:59:04.816722 44149 nvc_info.c:352] missing ipc path /tmp/nvidia-mps I1115 20:59:04.816727 44149 nvc_info.c:854] requesting device information with '' I1115 20:59:04.823516 44149 nvc_info.c:745] listing device /dev/nvidia0 (GPU-adc95203-e773-8d6f-cd05-68b9c4c018d2 at 00000000:52:00.0) NVRM version: 535.104.12 CUDA version: 12.2

Device Index: 0 Device Minor: 0 Model: NVIDIA RTX 6000 Ada Generation Brand: NvidiaRTX GPU UUID: GPU-adc95203-e773-8d6f-cd05-68b9c4c018d2 Bus Location: 00000000:52:00.0 Architecture: 8.9 I1115 20:59:04.823644 44149 nvc.c:434] shutting down library context I1115 20:59:04.823820 44152 rpc.c:95] terminating nvcgo rpc service I1115 20:59:04.824737 44149 rpc.c:135] nvcgo rpc service terminated successfully I1115 20:59:04.828945 44151 rpc.c:95] terminating driver rpc service I1115 20:59:04.829273 44149 rpc.c:135] driver rpc service terminated successfully

 - [x] Kernel version from `uname -a`
 - [x] Any relevant kernel output lines from `dmesg`

(base) mona@ada:~$ sudo dmesg | grep docker [ 18.640427] audit: type=1400 audit(1700077105.624:68): apparmor="STATUS" operation="profile_load" profile="unconfined" name="docker-default" pid=1944 comm="apparmor_parser"

 - [ ] Driver information from `nvidia-smi -a`

(base) mona@ada:~$ nvidia-smi -a

==============NVSMI LOG==============

Timestamp : Wed Nov 15 16:00:24 2023 Driver Version : 535.104.12 CUDA Version : 12.2

Attached GPUs : 1 GPU 00000000:52:00.0 Product Name : NVIDIA RTX 6000 Ada Generation Product Brand : NVIDIA RTX Product Architecture : Ada Lovelace Display Mode : Enabled Display Active : Enabled Persistence Mode : Enabled Addressing Mode : None MIG Mode Current : N/A Pending : N/A Accounting Mode : Disabled Accounting Mode Buffer Size : 4000 Driver Model Current : N/A Pending : N/A Serial Number : 1321923015834 GPU UUID : GPU-adc95203-e773-8d6f-cd05-68b9c4c018d2 Minor Number : 0 VBIOS Version : 95.02.48.00.08 MultiGPU Board : No Board ID : 0x5200 Board Part Number : 900-5G133-2750-001 GPU Part Number : 26B1-870-A1 FRU Part Number : N/A Module ID : 1 Inforom Version Image Version : G133.0510.00.01 OEM Object : 2.1 ECC Object : 6.16 Power Management Object : N/A GPU Operation Mode Current : N/A Pending : N/A GSP Firmware Version : N/A GPU Virtualization Mode Virtualization Mode : None Host VGPU Mode : N/A GPU Reset Status Reset Required : No Drain and Reset Recommended : N/A IBMNPU Relaxed Ordering Mode : N/A PCI Bus : 0x52 Device : 0x00 Domain : 0x0000 Device Id : 0x26B110DE Bus Id : 00000000:52:00.0 Sub System Id : 0x16A117AA GPU Link Info PCIe Generation Max : 4 Current : 1 Device Current : 1 Device Max : 4 Host Max : 5 Link Width Max : 16x Current : 16x Bridge Chip Type : N/A Firmware : N/A Replays Since Reset : 0 Replay Number Rollovers : 0 Tx Throughput : 2000 KB/s Rx Throughput : 16000 KB/s Atomic Caps Inbound : N/A Atomic Caps Outbound : N/A Fan Speed : 30 % Performance State : P8 Clocks Event Reasons Idle : Active Applications Clocks Setting : Not Active SW Power Cap : Not Active HW Slowdown : Not Active HW Thermal Slowdown : Not Active HW Power Brake Slowdown : Not Active Sync Boost : Not Active SW Thermal Slowdown : Not Active Display Clock Setting : Not Active FB Memory Usage Total : 49140 MiB Reserved : 515 MiB Used : 1307 MiB Free : 47317 MiB BAR1 Memory Usage Total : 65536 MiB Used : 29 MiB Free : 65507 MiB Conf Compute Protected Memory Usage Total : 0 MiB Used : 0 MiB Free : 0 MiB Compute Mode : Default Utilization Gpu : 25 % Memory : 10 % Encoder : 0 % Decoder : 0 % JPEG : 0 % OFA : 0 % Encoder Stats Active Sessions : 0 Average FPS : 0 Average Latency : 0 FBC Stats Active Sessions : 0 Average FPS : 0 Average Latency : 0 ECC Mode Current : Disabled Pending : Disabled ECC Errors Volatile SRAM Correctable : N/A SRAM Uncorrectable : N/A DRAM Correctable : N/A DRAM Uncorrectable : N/A Aggregate SRAM Correctable : N/A SRAM Uncorrectable : N/A DRAM Correctable : N/A DRAM Uncorrectable : N/A Retired Pages Single Bit ECC : N/A Double Bit ECC : N/A Pending Page Blacklist : N/A Remapped Rows Correctable Error : 0 Uncorrectable Error : 0 Pending : No Remapping Failure Occurred : No Bank Remap Availability Histogram Max : 192 bank(s) High : 0 bank(s) Partial : 0 bank(s) Low : 0 bank(s) None : 0 bank(s) Temperature GPU Current Temp : 44 C GPU T.Limit Temp : 41 C GPU Shutdown T.Limit Temp : -7 C GPU Slowdown T.Limit Temp : -2 C GPU Max Operating T.Limit Temp : 0 C GPU Target Temperature : 85 C Memory Current Temp : N/A Memory Max Operating T.Limit Temp : N/A GPU Power Readings Power Draw : 31.00 W Current Power Limit : 300.00 W Requested Power Limit : 300.00 W Default Power Limit : 300.00 W Min Power Limit : 100.00 W Max Power Limit : 300.00 W Module Power Readings Power Draw : N/A Current Power Limit : N/A Requested Power Limit : N/A Default Power Limit : N/A Min Power Limit : N/A Max Power Limit : N/A Clocks Graphics : 405 MHz SM : 405 MHz Memory : 405 MHz Video : 1185 MHz Applications Clocks Graphics : 2505 MHz Memory : 10001 MHz Default Applications Clocks Graphics : 2505 MHz Memory : 10001 MHz Deferred Clocks Memory : N/A Max Clocks Graphics : 3105 MHz SM : 3105 MHz Memory : 10001 MHz Video : 2415 MHz Max Customer Boost Clocks Graphics : N/A Clock Policy Auto Boost : N/A Auto Boost Default : N/A Voltage Graphics : 905.000 mV Fabric State : N/A Status : N/A Processes GPU instance ID : N/A Compute instance ID : N/A Process ID : 2254 Type : G Name : /usr/lib/xorg/Xorg Used GPU Memory : 464 MiB GPU instance ID : N/A Compute instance ID : N/A Process ID : 2453 Type : G Name : /usr/bin/gnome-shell Used GPU Memory : 40 MiB GPU instance ID : N/A Compute instance ID : N/A Process ID : 2945 Type : G Name : /usr/share/teams/teams --type=gpu-process --field-trial-handle=14231588369440436615,8501862656245088743,131072 --enable-features=ContextBridgeMutability,WebComponentsV0Enabled --disable-features=CookiesWithoutSameSiteMustBeSecure,SameSiteByDefaultCookies,SpareRendererForSitePerProcess --enable-crash-reporter=d50b8966-71b9-4d77-a50d-59ac0d5209c2,no_channel --global-crash-keys=d50b8966-71b9-4d77-a50d-59ac0d5209c2,no_channel,_companyName=Microsoft,_productName=com.microsoft.teams.linux,_version=1.5.00.10453 --gpu-preferences=MAAAAAAAAAAgAAAQAAAAAAAAAAAAAAAAAABgAAAAAAAQAAAAAAAAAAAAAAAAAAAACAAAAAAAAAA= --shared-files Used GPU Memory : 117 MiB GPU instance ID : N/A Compute instance ID : N/A Process ID : 6355 Type : G Name : /usr/share/code/code --type=gpu-process --crashpad-handler-pid=6340 --enable-crash-reporter=a1d28002-efec-451c-b87b-aa9cfca6c4eb,no_channel --user-data-dir=/home/mona/.config/Code --gpu-preferences=WAAAAAAAAAAgAAAEAAAAAAAAAAAAAAAAAABgAAAAAAA4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAGAAAAAAAAAAYAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAA== --shared-files --field-trial-handle=0,i,14230463884700741152,5169961603588991114,262144 --disable-features=CalculateNativeWinOcclusion,SpareRendererForSitePerProcess Used GPU Memory : 102 MiB GPU instance ID : N/A Compute instance ID : N/A Process ID : 28906 Type : G Name : /snap/firefox/3358/usr/lib/firefox/firefox Used GPU Memory : 229 MiB GPU instance ID : N/A Compute instance ID : N/A Process ID : 35075 Type : G Name : blender Used GPU Memory : 321 MiB

 - [x] Docker version from `docker version`
 - [x] NVIDIA packages version from `dpkg -l '*nvidia*'` _or_ `rpm -qa '*nvidia*'`

(base) mona@ada:~$ dpkg -l 'nvidia' Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-===================================-===========================-============-========================================================= un libgldispatch0-nvidia (no description available) ii libnvidia-cfg1-535:amd64 535.104.12-0ubuntu1 amd64 NVIDIA binary OpenGL/GLX configuration library un libnvidia-cfg1-any (no description available) un libnvidia-common (no description available) ii libnvidia-common-535 535.104.12-0ubuntu1 all Shared files used by the NVIDIA libraries un libnvidia-compute (no description available) rc libnvidia-compute-515:amd64 525.125.06-0ubuntu0.22.04.1 amd64 Transitional package for libnvidia-compute-525 rc libnvidia-compute-525:amd64 525.125.06-0ubuntu0.22.04.1 amd64 NVIDIA libcompute package ii libnvidia-compute-535:amd64 535.104.12-0ubuntu1 amd64 NVIDIA libcompute package ii libnvidia-compute-535:i386 535.104.12-0ubuntu1 i386 NVIDIA libcompute package ii libnvidia-container-tools 1.14.3-1 amd64 NVIDIA container runtime library (command-line tools) ii libnvidia-container1:amd64 1.14.3-1 amd64 NVIDIA container runtime library un libnvidia-decode (no description available) ii libnvidia-decode-535:amd64 535.104.12-0ubuntu1 amd64 NVIDIA Video Decoding runtime libraries ii libnvidia-decode-535:i386 535.104.12-0ubuntu1 i386 NVIDIA Video Decoding runtime libraries ii libnvidia-egl-wayland1:amd64 1:1.1.9-1.1 amd64 Wayland EGL External Platform library -- shared library un libnvidia-encode (no description available) ii libnvidia-encode-535:amd64 535.104.12-0ubuntu1 amd64 NVENC Video Encoding runtime library ii libnvidia-encode-535:i386 535.104.12-0ubuntu1 i386 NVENC Video Encoding runtime library un libnvidia-encode1 (no description available) un libnvidia-extra (no description available) ii libnvidia-extra-535:amd64 535.104.12-0ubuntu1 amd64 Extra libraries for the NVIDIA driver un libnvidia-fbc1 (no description available) ii libnvidia-fbc1-535:amd64 535.104.12-0ubuntu1 amd64 NVIDIA OpenGL-based Framebuffer Capture runtime library ii libnvidia-fbc1-535:i386 535.104.12-0ubuntu1 i386 NVIDIA OpenGL-based Framebuffer Capture runtime library un libnvidia-gl (no description available) un libnvidia-gl-390 (no description available) un libnvidia-gl-410 (no description available) ii libnvidia-gl-535:amd64 535.104.12-0ubuntu1 amd64 NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD ii libnvidia-gl-535:i386 535.104.12-0ubuntu1 i386 NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD un libnvidia-legacy-390xx-egl-wayland1 (no description available) un libnvidia-ml.so.1 (no description available) un libnvidia-ml1 (no description available) un nvidia-384 (no description available) un nvidia-390 (no description available) un nvidia-common (no description available) un nvidia-compute-utils (no description available) rc nvidia-compute-utils-525 525.125.06-0ubuntu0.22.04.1 amd64 NVIDIA compute utilities ii nvidia-compute-utils-535 535.104.12-0ubuntu1 amd64 NVIDIA compute utilities un nvidia-container-runtime (no description available) un nvidia-container-runtime-hook (no description available) ii nvidia-container-toolkit 1.14.3-1 amd64 NVIDIA Container toolkit ii nvidia-container-toolkit-base 1.14.3-1 amd64 NVIDIA Container Toolkit Base rc nvidia-dkms-525 525.125.06-0ubuntu0.22.04.1 amd64 NVIDIA DKMS package ii nvidia-dkms-535 535.104.12-0ubuntu1 amd64 NVIDIA DKMS package un nvidia-dkms-kernel (no description available) un nvidia-docker (no description available) ii nvidia-docker2 2.13.0-1 all nvidia-docker CLI wrapper ii nvidia-driver-535 535.104.12-0ubuntu1 amd64 NVIDIA driver metapackage un nvidia-driver-binary (no description available) un nvidia-egl-wayland-common (no description available) un nvidia-kernel-common (no description available) rc nvidia-kernel-common-525 525.125.06-0ubuntu0.22.04.1 amd64 Shared files used with the kernel module ii nvidia-kernel-common-535 535.104.12-0ubuntu1 amd64 Shared files used with the kernel module un nvidia-kernel-open (no description available) un nvidia-kernel-open-535 (no description available) un nvidia-kernel-source (no description available) un nvidia-kernel-source-525 (no description available) ii nvidia-kernel-source-535 535.104.12-0ubuntu1 amd64 NVIDIA kernel source package un nvidia-libopencl1-dev (no description available) ii nvidia-modprobe 545.23.06-0ubuntu1 amd64 Load the NVIDIA kernel driver and create device files un nvidia-opencl-icd (no description available) un nvidia-persistenced (no description available) ii nvidia-prime 0.8.17.1 all Tools to enable NVIDIA's Prime ii nvidia-settings 545.23.06-0ubuntu1 amd64 Tool for configuring the NVIDIA graphics driver un nvidia-settings-binary (no description available) un nvidia-smi (no description available) un nvidia-utils (no description available) ii nvidia-utils-535 535.104.12-0ubuntu1 amd64 NVIDIA driver support binaries ii xserver-xorg-video-nvidia-535 535.104.12-0ubuntu1 amd64 NVIDIA binary Xorg driver

 - [x] NVIDIA container library version from `nvidia-container-cli -V`

(base) mona@ada:~$ nvidia-container-cli -V cli-version: 1.14.3 lib-version: 1.14.3 build date: 2023-10-19T11:32+00:00 build revision: 1eb5a30a6ad0415550a9df632ac8832bf7e2bbba build compiler: x86_64-linux-gnu-gcc-7 7.5.0 build platform: x86_64 build flags: -D_GNU_SOURCE -D_FORTIFY_SOURCE=2 -DNDEBUG -std=gnu11 -O2 -g -fdata-sections -ffunction-sections -fplan9-extensions -fstack-protector -fno-strict-aliasing -fvisibility=hidden -Wall -Wextra -Wcast-align -Wpointer-arith -Wmissing-prototypes -Wnonnull -Wwrite-strings -Wlogical-op -Wformat=2 -Wmissing-format-attribute -Winit-self -Wshadow -Wstrict-prototypes -Wunreachable-code -Wconversion -Wsign-conversion -Wno-unknown-warning-option -Wno-format-extra-args -Wno-gnu-alignof-expression -Wl,-zrelro -Wl,-znow -Wl,-zdefs -Wl,--gc-sections


 - [ ] NVIDIA container library logs (see [troubleshooting](https://github.com/NVIDIA/nvidia-docker/wiki/Troubleshooting))
 - [x] Docker command, image and tag used
monajalal commented 7 months ago

answered here https://askubuntu.com/questions/1492809/unable-to-find-image-nvidia-cuda11-0-base-locally-when-testing-nvidia-docker2