Support docker plugin push

[rootsongjc]

Docker 17.03 Harbor 0.4.5 Run this command to push a docker volume plugin docker plugin push harbor-host/library/sshfs:latest Got this error:

c08c951b53b7: Preparing 
denied: requested access to the resource is denied

However I try to push it into docker hub, it works correctly. I think it's a problem of Harbor which do not support docker plugin pushing.

[reasonerjt]

Could you provide the log files?

[reasonerjt]

all .log under /var/log/harbor/${the date you see problem}

[rootsongjc]

docker_proxy.log

Mar  7 18:01:59 172.17.0.1 docker/proxy[107164]: 172.26.126.73 - - [07/Mar/2017:10:01:59 +0000] "GET /v2/ HTTP/1.1" 401 87 "-" "docker/17.03.0-ce go/go1.7.5 git-commit/3a232c8 kernel/4.9.12-moby os/linux arch/amd64 UpstreamClient(Docker-Client/17.03.0-ce \x5C(darwin\x5C))"
Mar  7 18:02:00 172.17.0.1 docker/proxy[107164]: 172.26.126.73 - admin [07/Mar/2017:10:02:00 +0000] "GET /service/token?account=admin&scope=repository%28plugin%29%3Alibrary%2Fsshfs%3Apush%2Cpull&service=token-service HTTP/1.1" 200 1184 "-" "docker/17.03.0-ce go/go1.7.5 git-commit/3a232c8 kernel/4.9.12-moby os/linux arch/amd64 UpstreamClient(Docker-Client/17.03.0-ce \x5C(darwin\x5C))"
Mar  7 18:02:00 172.17.0.1 docker/proxy[107164]: 172.26.126.73 - - [07/Mar/2017:10:02:00 +0000] "POST /v2/library/sshfs/blobs/uploads/ HTTP/1.1" 401 206 "-" "docker/17.03.0-ce go/go1.7.5 git-commit/3a232c8 kernel/4.9.12-moby os/linux arch/amd64 UpstreamClient(Docker-Client/17.03.0-ce \x5C(darwin\x5C))"

docker_registry.log

Mar  7 18:02:00 172.17.0.1 docker/registry[107164]: time="2017-03-07T10:02:00.206165331Z" level=warning msg="error authorizing context: insufficient scope" go.version=go1.6.3 http.request.host=sz-pg-oam-docker-hub-001.tendcloud.com http.request.id=474f8b87-301f-4763-995c-15758bee9995 http.request.method=POST http.request.remoteaddr=172.26.126.73 http.request.uri="/v2/library/sshfs/blobs/uploads/" http.request.useragent="docker/17.03.0-ce go/go1.7.5 git-commit/3a232c8 kernel/4.9.12-moby os/linux arch/amd64 UpstreamClient(Docker-Client/17.03.0-ce \\(darwin\\))" instance.id=192f21a4-bbf1-4c45-9dab-b154ce6626dd service=registry vars.name="library/sshfs" version=v2.5.0
Mar  7 18:02:00 172.17.0.1 docker/registry[107164]: 172.17.0.7 - - [07/Mar/2017:10:02:00 +0000] "POST /v2/library/sshfs/blobs/uploads/ HTTP/1.1" 401 206 "" "docker/17.03.0-ce go/go1.7.5 git-commit/3a232c8 kernel/4.9.12-moby os/linux arch/amd64 UpstreamClient(Docker-Client/17.03.0-ce \\(darwin\\))"

docker_ui.log

Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017-03-07T10:02:00Z [DEBUG] [authutils.go:67]: scopes: [repository(plugin):library/sshfs:push,pull]
Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017-03-07T10:02:00Z [INFO] request url: /service/token?account=admin&scope=repository%28plugin%29%3Alibrary%2Fsshfs%3Apush%2Cpull&service=token-service
Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017-03-07T10:02:00Z [WARNING] Failed to get secret cookie, error: http: named cookie not present
Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017-03-07T10:02:00Z [DEBUG] [token.go:53]: uid for logging: admin
Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017-03-07T10:02:00Z [DEBUG] [authenticator.go:57]: Current AUTH_MODE is db_auth
Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017-03-07T10:02:00Z [DEBUG] [token.go:63]: username for filtering access: admin.
Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017-03-07T10:02:00Z [INFO] current access, type: repository(plugin), name:library/sshfs, actions:[]
Mar  7 18:02:00 172.17.0.1 docker/ui[107164]: 2017/03/07 10:02:00 #033[1;34m[router.go:829][D] | GET        | /service/token                           | 51.132156ms      | match      | /service/token                           | #033[0m

[reasonerjt]

@rootsongjc could you also share me the link you mentioned in wechat group?
I think I lost it. It'd be helpful to put all information in the issue.

[rootsongjc]

https://rootsongjc.github.io/docker-practice/docs/plugin_developing.html

[mattzuba]

It looks like all that's needed is to modify the registryFilterMap in https://github.com/vmware/harbor/blob/master/src/ui/service/token/creator.go#L45 to account for repository(plugin), which, according to https://docs.docker.com/registry/spec/auth/scope/#resource-type, should be treated as a repository type. Or am I missing something else?

[reasonerjt]

Please also consider the signature/vuln scan scenarios. i.e. make sure the digest of a plugin matches in Notary and scan result stored in Harbor.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kumy]

Hi, do you have a milestone for this feature? Thanks!

[green1893]

Got the same probleam, already login by user admin and docker images could be pushed success.

there is command i tried.

docker plugin install --alias x.x.x.x/rexray/rbd rexray/rbd docker plugin push x.x.x.x/rexray/rbd

project rexray in harbor is created before push

Harbor installed with harbor-offline-installer-v1.7.1.tgz, all config is in default

version info:

Client:
 Version:           18.09.2
 API version:       1.39
 Go version:        go1.10.6
 Git commit:        6247962
 Built:             Sun Feb 10 04:13:27 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.2
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.6
  Git commit:       6247962
  Built:            Sun Feb 10 03:47:25 2019
  OS/Arch:          linux/amd64
  Experimental:     false

More logs in harbor:

proxy.log

Feb 20 16:23:22 172.18.0.1 proxy[20877]: 192.168.3.115 - "GET /v2/ HTTP/1.1" 401 87 "-" "docker/1.13.1 go/go1.7.5 git-commit/092cba3 kernel/3.10.0-693.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \x5C(linux\x5C))" 0.008 0.008 . Feb 20 16:23:22 172.18.0.1 proxy[20877]: 192.168.3.115 - "GET /service/token?account=admin&scope=repository%28plugin%29%3Arexray%2Frbd%3Apush%2Cpull&service=harbor-registry HTTP/1.1" 200 950 "-" "docker/1.13.1go/go1.7.5 git-commit/092cba3 kernel/3.10.0-693.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \x5C(linux\x5C))" 0.045 0.045 . Feb 20 16:23:23 172.18.0.1 proxy[20877]: 192.168.3.115 - "POST /v2/rexray/rbd/blobs/uploads/ HTTP/1.1" 401 222 "-" "docker/1.13.1 go/go1.7.5 git-commit/092cba3 kernel/3.10.0-693.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \x5C(linux\x5C))" 0.005 0.005 .

registry.log

Feb 20 16:23:22 172.18.0.1 registry[20877]: time="2019-02-20T08:23:22.871494171Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.7.3 http.request.host=192.168.2.150 http.request.id=6c2faa8e-68fd-4ef8-b202-9a991ec92144 http.request.method=GET http.request.remoteaddr=192.168.3.115 http.request.uri="/v2/" http.request.useragent="docker/1.13.1 go/go1.7.5 git-commit/092cba3 kernel/3.10.0-693.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \\(linux\\))" instance.id=758f7602-b302-41d2-a601-be0b4e85763f service=registry version=v2.6.2 Feb 20 16:23:22 172.18.0.1 registry[20877]: 172.18.0.8 - - [20/Feb/2019:08:23:22 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/1.13.1 go/go1.7.5 git-commit/092cba3 kernel/3.10.0-693.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \\(linux\\))" Feb 20 16:23:23 172.18.0.1 registry[20877]: time="2019-02-20T08:23:23.002457768Z" level=warning msg="error authorizing context: insufficient scope" go.version=go1.7.3 http.request.host=192.168.2.150 http.request.id=0feacc26-a15b-4184-bc32-165fe28cc9d6 http.request.method=POST http.request.remoteaddr=192.168.3.115 http.request.uri="/v2/rexray/rbd/blobs/uploads/" http.request.useragent="docker/1.13.1 go/go1.7.5 git-commit/092cba3 kernel/3.10.0-693.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \\(linux\\))" instance.id=758f7602-b302-41d2-a601-be0b4e85763f service=registry vars.name="rexray/rbd" version=v2.6.2 Feb 20 16:23:23 172.18.0.1 registry[20877]: 172.18.0.8 - - [20/Feb/2019:08:23:23 +0000] "POST /v2/rexray/rbd/blobs/uploads/ HTTP/1.1" 401 222 "" "docker/1.13.1 go/go1.7.5 git-commit/092cba3 kernel/3.10.0-693.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \\(linux\\))"

core.log

Feb 20 16:23:22 172.18.0.1 core[20877]: 2019/02/20 08:23:22 #033[1;44m[D] [server.go:2619] | 192.168.3.115|#033[43m 401 #033[0m| 6.657192ms| match|#033[44m GET #033[0m /v2/ r:/v2/*#033[0m Feb 20 16:23:22 172.18.0.1 core[20877]: 2019-02-20T08:23:22Z [WARNING] No filter found for access type: repository(plugin), skip filter, the access of resource 'rexray/rbd' will be set empty. Feb 20 16:23:22 172.18.0.1 core[20877]: 2019/02/20 08:23:22 #033[1;44m[D] [server.go:2619] | 192.168.3.115|#033[42m 200 #033[0m| 42.135321ms| match|#033[44m GET #033[0m /service/token r:/service/token#033[0m Feb 20 16:23:23 172.18.0.1 core[20877]: 2019/02/20 08:23:23 #033[1;44m[D] [server.go:2619] | 192.168.3.115|#033[43m 401 #033[0m| 3.645326ms| match|#033[46m POST #033[0m /v2/rexray/rbd/blobs/uploads/ r:/v2/*#033[0m

[ziggekatten]

Pushing for this as well, as we have air gapped systems that makes us need to place storage plugins in our private harbor repo

[anasypany]

Pushing for this as well, as we have air gapped systems that makes us need to place storage plugins in our private harbor repo

We are in the exact same boat here. +1 for an extremely useful feature for those of us with complete offline deployments.

[michmike]

@xaleeks will this scenario work when Harbor is OCI compliant?

[crazy-canux]

any update?

[michmike]

@xaleeks would know.

[ziggekatten]

Any update on this?

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[denis-accesa]

still relevant