goharbor / harbor

An open source trusted cloud native registry project that stores, signs, and scans content.
https://goharbor.io
Apache License 2.0
23.81k stars 4.74k forks source link

swift backend storage error #792

Closed biolounge closed 8 years ago

biolounge commented 8 years ago

Using swift backend as storage for registry is failing with following error (registry.log).

Sep 12 08:32:49 172.18.0.1 registry[1433]: time="2016-09-12T12:32:49.064356273Z" level=info msg="debug server listening localhost:5001" Sep 12 08:32:49 172.18.0.1 registry[1433]: panic: Swift authentication failed: Response didn't have storage url and auth token Sep 12 08:32:49 172.18.0.1 registry[1433]: Sep 12 08:32:49 172.18.0.1 registry[1433]: goroutine 1 [running]: Sep 12 08:32:49 172.18.0.1 registry[1433]: panic(0xd88060, 0xc820011c40) Sep 12 08:32:49 172.18.0.1 registry[1433]: #011/usr/local/go/src/runtime/panic.go:481 +0x3e6 Sep 12 08:32:49 172.18.0.1 registry[1433]: github.com/docker/distribution/registry/handlers.NewApp(0x7f1e5ae5 3680, 0xc8203d2300, 0xc820384580, 0x7f1e5ae53680) Sep 12 08:32:49 172.18.0.1 registry[1433]: #011/go/src/github.com/docker/distribution/registry/handlers/app.g o:121 +0x619 Sep 12 08:32:49 172.18.0.1 registry[1433]: github.com/docker/distribution/registry.NewRegistry(0x7f1e5ae536c0 , 0xc8203d2300, 0xc820384580, 0xe, 0x0, 0x0) Sep 12 08:32:49 172.18.0.1 registry[1433]: #011/go/src/github.com/docker/distribution/registry/registry.go:86 +0x2b0 Sep 12 08:32:49 172.18.0.1 registry[1433]: github.com/docker/distribution/registry.glob.func1(0x14ccdc0, 0xc8 203bc520, 0x1, 0x1) Sep 12 08:32:49 172.18.0.1 registry[1433]: #011/go/src/github.com/docker/distribution/registry/registry.go:55 +0x2a9 Sep 12 08:32:49 172.18.0.1 registry[1433]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(_Command).execute(0x14ccdc0, 0xc8203bc4d0, 0x1, 0x1, 0x0, 0x0) Sep 12 08:32:49 172.18.0.1 registry[1433]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x6d4 Sep 12 08:32:49 172.18.0.1 registry[1433]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(_Command).Execute(0x14ccf60, 0x0, 0x0) Sep 12 08:32:49 172.18.0.1 registry[1433]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x180 Sep 12 08:32:49 172.18.0.1 registry[1433]: main.main() Sep 12 08:32:49 172.18.0.1 registry[1433]: #011/go/src/github.com/docker/distribution/cmd/registry/main.go:23 +0x23

We looked in the swift logs and there are no authentication errors.

See the below keystone logs: 2016-09-12 14:52:04.863 24657 INFO keystone.common.wsgi [req-34a5a1fa-ba22-43f5-a640-be037db8a7a9 - - - - -] POST http://xxxxx.xxx.com:5000/v3/auth/tokens 2016-09-12 14:52:05.064 24657 INFO keystone.token.providers.fernet.utils [req-34a5a1fa-ba22-43f5-a640-be037db8a7a9 - - - - -] Loaded 2 encryption keys (max_active_keys=3) from: /etc/keystone/fernet-keys/

Let me know if you need more information.


docker version

Client: Version: 1.12.1 API version: 1.24 Go version: go1.6.3 Git commit: 23cf638 Built: OS/Arch: linux/amd64

Server: Version: 1.12.1 API version: 1.24 Go version: go1.6.3 Git commit: 23cf638 Built: OS/Arch: linux/amd64

docker-compose -version

docker-compose version 1.8.0, build f3628c7

hainingzhang commented 8 years ago

@biolounge The swift storage is provided by Docker distribution. Could you provide the configuration of swift backend? Have you configured something similar to our guide https://github.com/vmware/harbor/blob/master/docs/installation_guide.md ?

Need to replace the username/password and keystone endpoint.

storage: swift: username: admin password: ADMIN_PASS authurl: http://keystone_addr:35357/v3 tenant: admin domain: default region: regionOne container: docker_images

biolounge commented 8 years ago

We followed the guide you mentioned above. Please see the swift backend configuration.

version: 0.1 log: level: debug fields: service: registry storage: cache: layerinfo: inmemory swift: username: admin password: **** authurl: http://xxxx.xxxx.xxx:5000/v3 tenant: admin domain: default insecureskipverify: true region: regionOne container: docker_images

maintenance: uploadpurging: enabled: false delete: enabled: true

admin account and tenant are available in Openstack keystone.

int32bit commented 8 years ago

@biolounge I use keystone v2, it woks. I am not sure if v3 can work. This is my config:

version: 0.1
log:
  fields:
    service: registry
storage:
  swift:
    tenant: admin
    username: admin
    password: 5c8d15f732104053
    authurl: http://10.0.10.216:5000/v2.0
    container: docker_registry
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

Be sure that the swift container docker_registry is created before you start registry/harbor service .

hainingzhang commented 8 years ago

@biolounge for swift v3, the auth URL may look like: https://storage.myprovider.com/v3/auth

For more info, refer to https://docs.docker.com/registry/configuration/

biolounge commented 8 years ago

Yes, that solved the issue!