NVIDIA-Merlin / HugeCTR

HugeCTR is a high efficiency GPU framework designed for Click-Through-Rate (CTR) estimating training
Apache License 2.0
937 stars 200 forks source link

build docker failed with 401 Unauthorized (Set Up the Development Environment With Merlin Containers) #434

Open SeekPoint opened 9 months ago

SeekPoint commented 9 months ago

With the guild of Set Up the Development Environment With Merlin Containers ==

https://nvidia-merlin.github.io/HugeCTR/main/hugectr_contributor_guide.html#set-up-the-development-environment-with-merlin-containers

/HugeCTR/Merlin/docker$ sudo docker build --pull -t merlin:ctr -f dockerfile.ctr --build-arg RELEASE=false --build-arg RMM_VER=vnightly --build-arg CUDF_VER=vnightly --build-arg NVTAB_VER=vnightly --build-arg HUGECTR_DEV_MODE=true --no-cache . [sudo] password for amd00: [+] Building 2.3s (7/7) FINISHED docker:default => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from dockerfile.ctr 0.0s => => transferring dockerfile: 3.54kB 0.0s => resolve image config for docker.io/docker/dockerfile:1.2 1.3s => CACHED docker-image://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc 0.0s => [internal] load build definition from dockerfile.ctr 0.0s => [internal] load .dockerignore 0.0s => ERROR [internal] load metadata for nvcr.io/nvstaging/merlin/merlin-base:23.06 0.8s

[internal] load metadata for nvcr.io/nvstaging/merlin/merlin-base:23.06:

dockerfile.ctr:1

1 | >>> # syntax=docker/dockerfile:1.2 2 | ARG MERLIN_VERSION=23.06 3 | ARG TRITON_VERSION=23.06

ERROR: failed to solve: pulling from host nvcr.io failed with status code [manifests 23.06]: 401 Unauthorized amd00@MZ32-00:~/yk_repo/HugeCTR/Merlin/docker$

EmmaQiaoCh commented 9 months ago

Hi @SeekPoint, Thanks for trying hugectr. Registry 'nvcr.io/nvstaging' can only be used internally. The merlin-base could be built from dockerfile.merlin(same path as dockerfile.ctr). Then pass the --build-arg BASE_IMAGE=$image_tag_built_from_dockerfile.merlin when docker build dockerfile.ctr.

SeekPoint commented 9 months ago

so, what the exact steps for set up a dev environments for external developer?

SeekPoint commented 9 months ago

I try to build HugeCTR from source on the image which I build with " Merlin/docker$ sudo docker build --pull -f dockerfile.merlin . " I it falied with

SeekPoint commented 9 months ago

the I try:

(base) amd00@MZ32-00:~/yk_repo/HugeCTR/Merlin/docker$ sudo docker build --pull -f dockerfile.ctr . [+] Building 3.6s (7/7) FINISHED docker:default => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from dockerfile.ctr 0.0s => => transferring dockerfile: 3.50kB 0.0s => resolve image config for docker.io/docker/dockerfile:1.2 2.0s => CACHED docker-image://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc 0.0s => [internal] load build definition from dockerfile.ctr 0.0s => [internal] load .dockerignore 0.0s => ERROR [internal] load metadata for docker.io/library/dockerfile.merlin:latest 1.4s

[internal] load metadata for docker.io/library/dockerfile.merlin:latest:

dockerfile.ctr:1

1 | >>> # syntax=docker/dockerfile:1.2 2 | ARG MERLIN_VERSION=23.06 3 | ARG TRITON_VERSION=23.06

ERROR: failed to solve: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed (base) amd00@MZ32-00:~/yk_repo/HugeCTR/Merlin/docker$