docker login is not useful to buildx

[Jinshijiming]

Pulling is successful.

root@ASME:/home/asme# docker login -u idaas_11ace77d89 ascendhub.huawei.com
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

docker pull ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1
22.0.RC3-1.8.1: Pulling from public-ascendhub/pytorch-modelzoo
Digest: sha256:8e7d76adbce1410ccff8e54e80e525820488650805daae51472ac6a9e25079ed
Status: Image is up to date for ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1
ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1

But,while using buildx,I get the error.

[+] Building 0.3s (3/3) FINISHED                                                                    
 => [internal] load build definition from Dockerfile                                           0.0s
 => => transferring dockerfile: 881B                                                           0.0s
 => [internal] load .dockerignore                                                              0.0s
 => => transferring context: 2B                                                                0.0s
 => ERROR [internal] load metadata for ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo  0.2s
------
 > [internal] load metadata for ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1:
------
Dockerfile:1
--------------------
   1 | >>> FROM ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1
   2 |     
   3 |     
--------------------
ERROR: failed to solve: ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1: pulling from host ascendhub.huawei.com failed with status code [manifests 22.0.RC3-1.8.1]: 403 Forbidden

[tonistiigi]

I'm not sure what registry you use but 403 usually does not mean login error, that would be 401

[Jinshijiming]

Thanks for your reply! I mean buildx may not be able to be a logged in state using docker, either docker driver or docfker container driver. However build can use logged in state, no 403 forbidden. Am I missing something?

[tonistiigi]

I'm saying that because this is 403, it is more likely that this error happens for some other reason than login credentials. For example, the registry is not spec compliant and can't handle some requests, or there is some proxy restriction in your networking somewhere etc. Debug logs from daemon side might show something.

[Jinshijiming]

Sorry, I'm a new docker. How can I access the daemon log? I'll check it carefully.

[rafrafek]

I have the same problem when pushing to eu.gcr.io -- it works with regular build but not with buildx (same error 403 Forbidden).

I found similar issue posted by someone else here: https://github.com/community/community/discussions/42889

[Jinshijiming]

680

[Jinshijiming]

@tonistiigi I got the log about buildx. The registry is ascend hub,could you have a try please？

docker -D --log-level "info" buildx build -t torch .

[+] Building 1.0s (3/3) FINISHED                                                                                                                
 => [internal] load build definition from Dockerfile                                                                                       0.0s
 => => transferring dockerfile: 32B                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                          0.0s
 => => transferring context: 2B                                                                                                            0.0s
 => ERROR [internal] load metadata for ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1                               1.0s
------
 > [internal] load metadata for ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo:22.0.RC3-1.8.1:
------
ERROR: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: unexpected status code [manifests 22.0.RC3-1.8.1]: 403 Forbidden
17227 v0.9.1-docker /usr/libexec/docker/cli-plugins/docker-buildx -D --log-level info buildx build -t torch .
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1
    /go/src/github.com/docker/buildx/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:72
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryClientInterceptor.func1
    /go/src/github.com/docker/buildx/vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/interceptor.go:99
github.com/moby/buildkit/client.filterInterceptor.func1
    /go/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/client/client.go:290
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1
    /go/src/github.com/docker/buildx/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:72
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1
    /go/src/github.com/docker/buildx/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:81
google.golang.org/grpc.(*ClientConn).Invoke
    /go/src/github.com/docker/buildx/vendor/google.golang.org/grpc/call.go:35
github.com/moby/buildkit/api/services/control.(*controlClient).Solve
    /go/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/api/services/control/control.pb.go:1535
github.com/moby/buildkit/client.(*Client).solve.func2
    /go/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/client/solve.go:231
golang.org/x/sync/errgroup.(*Group).Go.func1
    /go/src/github.com/docker/buildx/vendor/golang.org/x/sync/errgroup/errgroup.go:75
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1571

17227 v0.9.1-docker /usr/libexec/docker/cli-plugins/docker-buildx -D --log-level info buildx build -t torch .
github.com/moby/buildkit/client.(*Client).solve.func2
    /go/src/github.com/docker/buildx/vendor/github.com/moby/buildkit/client/solve.go:244
golang.org/x/sync/errgroup.(*Group).Go.func1
    /go/src/github.com/docker/buildx/vendor/golang.org/x/sync/errgroup/errgroup.go:75
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1571

[crazy-max]

@Jinshijiming Can you show the output of docker buildx ls please?

Also BuildKit logs would be more useful. You can create a container builder with debug enabled and post container logs here:

$ docker buildx create --name mybuilder --buildkitd-flags '--debug' --driver docker-container --use --bootstrap
$ docker buildx build ....
...
$ docker logs buildx_buildkit_mybuilder0