search

cloudfoundry / diego-release

BOSH Release for Diego
Apache License 2.0
201 stars 213 forks source link

crashed at github.com/docker/docker/registry.NewSession(0x0, 0x0, 0xc2081b4be0, 0x1, 0xc2081993b0, 0x0, 0x0) #34

Closed guanglinlv closed 9 years ago

guanglinlv commented 9 years ago

hi all,

I get a crash problem when i try to get docker image from my private docker registry.it is trying to access a null pointer.

here is the stack:

2015/04/20 21:04:45 http: panic serving 127.0.0.1:59200: runtime error: invalid memory address or nil pointer dereference
goroutine 947 [running]:
net/http.func·011()
    /usr/local/go/src/net/http/server.go:1130 +0xbb
github.com/docker/docker/registry.NewSession(0x0, 0x0, 0xc2081b2aa0, 0x1, 0xc208152b10, 0x0, 0x0)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/Godeps/_workspace/src/github.com/docker/docker/registry/session.go:58 +0x75a
github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher.registryProvider.ProvideRegistry(0xa49eb0, 0x1b, 0xc208103720, 0x1, 0x1, 0xc2080fc349, 0xf, 0x0, 0x0, 0x0, ...)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher/repository_provider.go:49 +0x2ff
github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher.(*registryProvider).ProvideRegistry(0xc2080a6cf0, 0xc2080fc349, 0xf, 0x0, 0x0, 0x0, 0x0)
    <autogenerated>:14 +0xe1
github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher.(*DockerRepositoryFetcher).Fetch(0xc2080a6d20, 0x7fa679df5a30, 0xc2081067e0, 0xc208096620, 0xc2080fc375, 0x6, 0x0, 0x0, 0x9ca0c0, 0x0, ...)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher/repository_fetcher.go:101 +0x324
github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher.Retryable.Fetch(0x7fa679df7350, 0xc2080a6d20, 0x7fa679df5a30, 0xc2081067e0, 0xc208096620, 0xc2080fc375, 0x6, 0x0, 0x0, 0x0, ...)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher/retryable.go:21 +0x15b
github.com/cloudfoundry-incubator/garden-linux/old/repository_fetcher.(*Retryable).Fetch(0xc208103730, 0x7fa679df5a30, 0xc2081067e0, 0xc208096620, 0xc2080fc375, 0x6, 0x0, 0x0, 0x7fa679de3000, 0x0, ...)
    <autogenerated>:15 +0x141
github.com/cloudfoundry-incubator/garden-linux/old/rootfs_provider.(*dockerRootFSProvider).ProvideRootFS(0xc20802f9a0, 0x7fa679df5a30, 0xc2081067e0, 0xc20814de10, 0xb, 0xc208096620, 0x0, 0x0, 0xc20816f1c8, 0x0, ...)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/old/rootfs_provider/docker_rootfs_provider.go:56 +0x13e
github.com/cloudfoundry-incubator/garden-linux/container_pool.(*LinuxContainerPool).acquireSystemResources(0xc2081681c0, 0xc20814de10, 0xb, 0xc2080964d0, 0x6e, 0xc2081364e0, 0x2d, 0xc2080fc340, 0x3b, 0xc208106780, ...)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/container_pool/container_pool.go:569 +0x6c9
github.com/cloudfoundry-incubator/garden-linux/container_pool.(*LinuxContainerPool).Create(0xc2081681c0, 0xc2080964d0, 0x6e, 0x34630b8a000, 0xc2080fc340, 0x3b, 0x0, 0x0, 0x0, 0x0, ...)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/container_pool/container_pool.go:254 +0x502
github.com/cloudfoundry-incubator/garden-linux/linux_backend.(*LinuxBackend).Create(0xc20802fa40, 0xc2080964d0, 0x6e, 0x34630b8a000, 0xc2080fc340, 0x3b, 0x0, 0x0, 0x0, 0x0, ...)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/linux_backend/linux_backend.go:147 +0x1b9
github.com/cloudfoundry-incubator/garden/server.(*GardenServer).handleCreate(0xc208064700, 0x7fa679df8cb8, 0xc20803f220, 0xc20817c4e0)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/Godeps/_workspace/src/github.com/cloudfoundry-incubator/garden/server/request_handling.go:61 +0x309
github.com/cloudfoundry-incubator/garden/server.*GardenServer.(github.com/cloudfoundry-incubator/garden/server.handleCreate)·fm(0x7fa679df8cb8, 0xc20803f220, 0xc20817c4e0)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/Godeps/_workspace/src/github.com/cloudfoundry-incubator/garden/server/server.go:74 +0x45
net/http.HandlerFunc.ServeHTTP(0xc2081962f0, 0x7fa679df8cb8, 0xc20803f220, 0xc20817c4e0)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/bmizerany/pat.(*PatternServeMux).ServeHTTP(0xc2080fa128, 0x7fa679df8cb8, 0xc20803f220, 0xc20817c4e0)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/Godeps/_workspace/src/github.com/bmizerany/pat/mux.go:109 +0x21c
github.com/cloudfoundry-incubator/garden/server.func·002(0x7fa679df8cb8, 0xc20803f220, 0xc20817c4e0)
    /var/vcap/packages/garden-linux/src/github.com/cloudfoundry-incubator/garden-linux/Godeps/_workspace/src/github.com/cloudfoundry-incubator/garden/server/server.go:113 +0x57
net/http.HandlerFunc.ServeHTTP(0xc208196550, 0x7fa679df8cb8, 0xc20803f220, 0xc20817c4e0)
    /usr/local/go/src/net/http/server.go:1265 +0x41
net/http.serverHandler.ServeHTTP(0xc208064710, 0x7fa679df8cb8, 0xc20803f220, 0xc20817c4e0)
    /usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc20812c1e0)
    /usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:1751 +0x35e
cf-gitbot commented 9 years ago

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/92807684.

guanglinlv commented 9 years ago

I have fixed the problem,it should add the condition: r.indexEndpoint.secure at github.com/docker/docker/registry.NewSession.

emalm commented 9 years ago

Hi, @guanglinlv,

Thanks for reporting the problem. This seems like an issue with garden-linux, which uses some Docker code internally to process images. Since the code that causes the exception is part of that vendored Docker code, it's not appropriate to alter it (although if there is a different version of the Docker code that does not exhibit this problem, we could potentially use that instead). I suspect that this is an issue with your configuration of garden-linux, though; could you please share your Diego deployment manifest (sanitized of sensitive credentials, of course) and which version (final BOSH release or GitHub commit SHA) of diego-release you have deployed?

Thanks, Eric, for the CF Runtime Diego team

guanglinlv commented 9 years ago

hi @ematpl ,I think the root cause is insecure docker registry configured in garden-linux.insecure_docker_registry_list

diego-release: diego-0.1102.0

deployment manifest:

compilation:
  cloud_properties:
    instance_type: m1.medium
  network: diego1
  reuse_compilation_vms: true
  workers: 6
director_uuid: 916a079c-7ef2-42c2-8206-881369eb0571
jobs:
- instances: 1
  name: etcd_z1
  networks:
  - name: diego1
    static_ips:
    - 10.10.10.131
  persistent_disk: 1024
  resource_pool: etcd_z1
  templates:
  - name: etcd
    release: diego
  update:
    max_in_flight: 1
    serial: true
- instances: 1
  name: brain_z1
  networks:
  - name: diego1
    static_ips:
    - 10.10.10.132
  properties:
    metron_agent:
      zone: z1
  resource_pool: brain_z1
  templates:
  - name: consul_agent
    release: cf
  - name: auctioneer
    release: diego
  - name: converger
    release: diego
  - name: runtime_metrics_server
    release: diego
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: true
- instances: 1
  name: cell_z1
  networks:
  - name: diego1
    static_ips:
    - 10.10.10.133
  properties:
    consul:
      agent:
        services:
        - receptor
    diego:
      rep:
        zone: z1
    metron_agent:
      zone: z1
  resource_pool: cell_z1
  templates:
  - name: rep
    release: diego
  - name: consul_agent
    release: cf
  - name: executor
    release: diego
  - name: garden-linux
    release: diego
  - name: receptor
    release: diego
  - name: ssh_proxy
    release: diego
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 1
  name: cc_bridge_z1
  networks:
  - name: diego1
    static_ips:
    - 10.10.10.134
  properties:
    consul:
      agent:
        services:
        - file_server
        - nsync
        - stager
        - tps
    metron_agent:
      zone: z1
  resource_pool: cc_bridge_z1
  templates:
  - name: stager
    release: diego
  - name: nsync
    release: diego
  - name: tps
    release: diego
  - name: file_server
    release: diego
  - name: consul_agent
    release: cf
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 1
  name: route_emitter_z1
  networks:
  - name: diego1
    static_ips:
    - 10.10.10.135
  properties:
    metron_agent:
      zone: z1
  resource_pool: route_emitter_z1
  templates:
  - name: route_emitter
    release: diego
  - name: consul_agent
    release: cf
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 0
  name: etcd_z2
  networks:
  - name: diego2
    static_ips: []
  persistent_disk: 1024
  resource_pool: etcd_z2
  templates:
  - name: etcd
    release: diego
  update:
    max_in_flight: 1
    serial: true
- instances: 0
  name: brain_z2
  networks:
  - name: diego2
  properties:
    metron_agent:
      zone: z2
  resource_pool: brain_z2
  templates:
  - name: consul_agent
    release: cf
  - name: auctioneer
    release: diego
  - name: converger
    release: diego
  - name: runtime_metrics_server
    release: diego
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: true
- instances: 0
  name: cell_z2
  networks:
  - name: diego2
  properties:
    consul:
      agent:
        services:
        - receptor
    diego:
      rep:
        zone: z2
    metron_agent:
      zone: z2
  resource_pool: cell_z2
  templates:
  - name: rep
    release: diego
  - name: consul_agent
    release: cf
  - name: executor
    release: diego
  - name: garden-linux
    release: diego
  - name: receptor
    release: diego
  - name: ssh_proxy
    release: diego
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 0
  name: cc_bridge_z2
  networks:
  - name: diego2
  properties:
    consul:
      agent:
        services:
        - file_server
        - nsync
        - stager
        - tps
    metron_agent:
      zone: z2
  resource_pool: cc_bridge_z2
  templates:
  - name: stager
    release: diego
  - name: nsync
    release: diego
  - name: tps
    release: diego
  - name: file_server
    release: diego
  - name: consul_agent
    release: cf
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 0
  name: route_emitter_z2
  networks:
  - name: diego2
  properties:
    metron_agent:
      zone: z2
  resource_pool: route_emitter_z2
  templates:
  - name: route_emitter
    release: diego
  - name: consul_agent
    release: cf
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 0
  name: etcd_z3
  networks:
  - name: diego3
    static_ips: []
  persistent_disk: 1024
  resource_pool: etcd_z3
  templates:
  - name: etcd
    release: diego
  update:
    max_in_flight: 1
    serial: true
- instances: 0
  name: brain_z3
  networks:
  - name: diego3
  properties:
    metron_agent:
      zone: z3
  resource_pool: brain_z3
  templates:
  - name: consul_agent
    release: cf
  - name: auctioneer
    release: diego
  - name: converger
    release: diego
  - name: runtime_metrics_server
    release: diego
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: true
- instances: 0
  name: cell_z3
  networks:
  - name: diego3
  properties:
    consul:
      agent:
        services:
        - receptor
    diego:
      rep:
        zone: z3
    metron_agent:
      zone: z3
  resource_pool: cell_z3
  templates:
  - name: rep
    release: diego
  - name: consul_agent
    release: cf
  - name: executor
    release: diego
  - name: garden-linux
    release: diego
  - name: receptor
    release: diego
  - name: ssh_proxy
    release: diego
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 0
  name: cc_bridge_z3
  networks:
  - name: diego3
  properties:
    consul:
      agent:
        services:
        - file_server
        - nsync
        - stager
        - tps
    metron_agent:
      zone: z3
  resource_pool: cc_bridge_z3
  templates:
  - name: stager
    release: diego
  - name: nsync
    release: diego
  - name: tps
    release: diego
  - name: file_server
    release: diego
  - name: consul_agent
    release: cf
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
- instances: 0
  name: route_emitter_z3
  networks:
  - name: diego3
  properties:
    metron_agent:
      zone: z3
  resource_pool: route_emitter_z3
  templates:
  - name: route_emitter
    release: diego
  - name: consul_agent
    release: cf
  - name: metron_agent
    release: cf
  update:
    max_in_flight: 1
    serial: false
name: cf-warden-diego
networks:
- cloud_properties: {}
  name: floating
  type: vip
- cloud_properties:
    gateway: 10.10.10.1
    net_id: 67a8ddc7-4d5f-432f-a154-660df2e8e69c
    security_groups:
    - default
  name: diego-dynamic
  type: dynamic
- name: diego1
  subnets:
  - cloud_properties:
      net_id: 67a8ddc7-4d5f-432f-a154-660df2e8e69c
      security_groups:
      - default
    gateway: 10.10.10.1
#    dns: [10.10.10.3]
    name: default_unused
    range: 10.10.10.0/24
    reserved:
    - 10.10.10.2 - 10.10.10.130
    static:
    - 10.10.10.131 - 10.10.10.155
  type: manual

- name: diego2
  subnets:
  - cloud_properties:
      net_id: 67a8ddc7-4d5f-432f-a154-660df2e8e69c
      security_groups:
      - default
    gateway: 10.10.10.1
    name: default_unused
    range: 10.10.10.0/24
    reserved:
    - 10.10.10.2 - 10.10.10.130
    static:
    - 10.10.10.131 - 10.10.10.155
  type: manual

- name: diego3
  subnets:
  - cloud_properties:
      net_id: 67a8ddc7-4d5f-432f-a154-660df2e8e69c
      security_groups:
      - default
    gateway: 10.10.10.1
    name: default_unused
    range: 10.10.10.0/24
    reserved:
    - 10.10.10.2 - 10.10.10.130
    static:
    - 10.10.10.131 - 10.10.10.155
  type: manual

properties:
  consul:
    agent:
      servers:
        lan:
        - 10.10.10.152
  diego:
    auctioneer:
      etcd:
        machines:
        - 10.10.10.131
      log_level: null
    converger:
      etcd:
        machines:
        - 10.10.10.131
      log_level: null
    etcd:
      machines:
      - 10.10.10.131
    executor:
      allow_privileged: null
      drain_timeout_in_seconds: 0
      garden:
        address: 127.0.0.1:7777
        network: tcp
      log_level: null
    file_server:
      cc:
        base_url: https://api.9.91.39.29.xip.io
        basic_auth_password: internal-password
        external_port: 9022
        staging_upload_password: upload-password
        staging_upload_user: upload-user
      log_level: null
    garden-linux:
      allow_networks: 
      - 9.91.0.0/16
      - 10.0.0.0/8
      disk_quota_enabled: false
      insecure_docker_registry_list: ["9.91.39.37:8080"]
      kernel_network_tuning_enabled: false
      listen_address: 0.0.0.0:7777
      listen_network: tcp
    nsync:
      cc:
        base_url: https://api.9.91.39.29.xip.io
        basic_auth_password: internal-password
        external_port: 9022
        staging_upload_password: upload-password
        staging_upload_user: upload-user
      diego_api_url: http://:@receptor.service.consul:8888
      etcd:
        machines:
        - 10.10.10.131
      log_level: null
    receptor:
      cors_enabled: null
      domain_names:
      - receptor.9.91.39.29.xip.io
      etcd:
        machines:
        - 10.10.10.131
      log_level: null
      nats:
        machines:
        - 10.10.10.142
        password: nats
        port: 4222
        username: nats
      password: ""
      register_with_router: true
      username: ""
    rep:
      etcd:
        machines:
        - 10.10.10.131
      log_level: null
    route_emitter:
      diego_api_url: http://:@receptor.service.consul:8888
      log_level: null
      nats:
        machines:
        - 10.10.10.142
        password: nats
        port: 4222
        username: nats
    runtime_metrics_server:
      diego_api_url: http://:@receptor.service.consul:8888
      etcd:
        machines:
        - 10.10.10.131
      log_level: null
      nats:
        machines:
        - 10.10.10.142
        password: nats
        port: 4222
        username: nats
    ssh_proxy:
      diego_api_url: http://:@receptor.service.consul:8888
      host_key: |+
        -----BEGIN RSA PRIVATE KEY-----
        MIIEhgIBAAKB/DMF5qOW+fh608KhX7qBLNHHmfzCfOONd176Oaf8rGht5KdnoNge
        TYSGqBFuYB1r1RbYEVhWAkH/8mW14XRVNmQ4C9eQDFqeWmmaOoSBG5GdP5GUfhI/
        z5vprQw+rnV4gt4InCA7QaR86pLj5sMiUij5OE/CW0dw29+z5E0p5WnQX5+utRmw
        ioQJD8jUDvzFrvzKIdE0HVOEl0agbeXq8U2e9E1de4iR+NiDc1zeiQmDNCIhFJb4
        FL7WqqokL+49SwSWGmOFKAlpj4Dlhx5dDwJWpcDe0XBXCkfcXn8xXNOT+4YBxJUG
        idNMPpLKpDUphZRj8CNBSMkjehIKVwIDAQABAoH8MiCAAQQYvXfeh36HT/IMmGSi
        8mIY1G5tclAfSNzCfS5Jz/XNXcYXnjW09LsdjoocJX9NOx30xeawvCA+SU5WS4uM
        htEscfLVHJ67EubMsPhuNZZPbZpnWuPucPM77ojg+UY4LKpKyVE4G+vvEJKtaTe/
        jQyDJOLKATL4/p5DtbDH7hVZcJVHU94csiE9a9OtyAvSwZLmNxGIBHshFntjcI+/
        hmQSFl3d1iduYGx7oeq3wX0sQ1mk/QksUTHRrlLfSQhLi5ZmH9Hnn/Qw2WeXKVdk
        BvXAUBiHG7Y0qGHXl5FOkB1BSlmk/EOkBk6gWl1a1Kx4A6oyNL4+HsuBAn572PqW
        IDutj4shf8ysI5fLJnvGCygZmk8LPZIlZZqLpDGo+l4iF3VCsd8CU2jKfWqel8+Q
        axdmu/BrQ7xyuWpxoHtKICv+CitI1ivzeYQwRCmjIN84jeGP9Pty4AJzhySegf/h
        n3irIp07wEzdedoj4A3RWWObX+AeubyUqfcCfml3scNb2oBK24RDVGYaUSWkSHBe
        OEU0QlOaJXZ2kCK2rIK/IVI7cD12WpkWTGY782VBmipEXwtMTprQzMrnK25shS+z
        AjCDGXtqr0GjxJh73WRurs1dVk6sqslSp1M/R9fmjGU4vdYL2JfMczEH4+57aOpR
        sW+H0FEYDayKoQJ+Eo8gdjDcYJT7N4jsRfuLesEImVQArV2HbNrMNNh2AWkYnAbw
        5lD3nIgFMFcJhBapTJzZWP4DYrzVOW3MJrEMd3yiHSiXDxm9BMw7h9/05DrCtpRt
        fw8b9zOyHrPdCiz9WteGXexE6/hi8ZpOqn3hJ7EiwPWRTK5gappQ3UJfAn4Tr0t2
        cwZtO4uNPCPcirzqkacTkgJeqEpY4ERtv+NXF1FLdfD6MC3ayuRN/mN0EWx0UbI8
        gVZb/XoOWzpeBJeOnKKfLIIUG+P9rQPY9IAVFclUnXPy0KDzPjcCLHMejokSOu2p
        VtXXxY4/huFZHWflcxM56NV9Q5QWDq8+rQECfjQTbNbd4ehbC/Q5EZ1SIzeaSLrn
        0ICmiRajnISbje5vPntqPXjBkbiVGx31qOaZ+DlGGLOyzW/GP5X4NOUwza2bYh3q
        nnzwBhoGLZfvoes5Nw06leOdVqcvIjLIDhb+XbiiEeAnONUp+BAKzDYOIp7K+LPe
        1rHeshh0P/QfCQ==
        -----END RSA PRIVATE KEY-----
      private_key: |+
        -----BEGIN RSA PRIVATE KEY-----
        MIIEowIBAAKCAQEAyl54KYG/DMelde36RZXJDtjoH9tXd6rb87605yLc4SUOg6b5
        0ou42hOEp/mo2u/HnH3X1u2swiIJamzqBxzbTAp+kDgJ69GNDf6FZwMwtlc478/i
        38rFsJ4tVyAFX8tabccKmdu4h+2igwINXiU1mlbZWme8s1wMm0O/HlsmizEDAgah
        DD2upA3s3VPTKQ2sBBmOO5oECjrVaU5EeD/snvZFqtl7zAp+kaRjjVe+8Dxz1fE8
        rPy2QP5Rc0VU9zwsznNe8d+c0DhC7KPqaMJNOFMNTpE8VxzLm0e/y0cmWZGyZWuY
        ov1y6HLitgUi6JwQOs72WIMkZCVpJ4NH75ghcwIDAQABAoIBAGdg4RObE0TtDEp1
        Op+nh+FMzy/O9hzMrLYUmwdkWN+u9X4to3t6IwFMbTz+HdNSIG+CHevO2K0HhDXY
        06i0hhaPWd5EslTEZMFcXzujdpP/dLeVRDm8tUanV1QlqHkqQdTIlFLK2radn6I7
        Pybz7TEd4sT7L3uUr1/1uMFCISuDAiv3jb81pmQWFpbSh9+xWQxktHwbbv/EhECy
        k8ha52bazVmXYIIIAHegw59bzYUzQD0rbM8aY1YqwxdKXOFCJHiCraTkUGh0pcbt
        XCfKKyeu24uxkO+om/vjeGkARFO5rD6vHaYiMA+h5PqSj57Mg6znPcjwjoFeahY5
        no6SvPECgYEA8T/Qj7PWiqV/GV8+GRTBT4T/dDeARE2sbXVSSdtC9sWaPw8AD2Gf
        R+fkQ/LT4Je9m688+qXpo1Wencvb2LYRtdNmPrgKoqIeWRM5Q6RggOR1KOLeVv/I
        BVT6Gwui63jdwSde9UFQvFXcmVRP5ceX90kdq6HTaE7LNYvgw+TGZHsCgYEA1r4T
        lXb+bHaLYUioQrzCLJmrBzzJC82gjR8Y11UY6wnkCqx7k4Jyt/ow1v+Rscn84yTN
        fedsfV0AVtjSIKi1RnkwLTiCaDtCtMr6kycJKMyDtOp4fnw2z25xw4PNREaEZWgU
        c1IjVz53omwt/f764yejzEqPp2RvFq0QSrTqUWkCgYBp2qpUo4A7japbI6NHtges
        MJs2DzRExCAoVxYoOgED7mhjyt0n4pnVHzbVabL2FR6pNRb4k9wUQvk6G7z3NcxH
        j96JHo/sFF5kjz7NI08Evzl5GzKb0deeH08ulsiBsaL7oZ7U1jfqq3IEBS6ZkB5p
        7rcKHBN67TfU4u9YhQVYCQKBgAR4KEgwFdOEFBqOPJJHA3OSLzETXsvUF8dYIlg4
        +S/LIOfn9xp22JRjrKiM4SdmINFJbs10B5oEWYF1OcpcOImi9BG831n0ActpfhWZ
        9mLW/tbBGojnZOykS9HZ6GfHX/IMXICZ8Qpbznme7sYejHI13P7ffmXpfTZnm7My
        WL4hAoGBAOuavq8+HUIRJCLUCl7o8ZKlJzleAxCpJFZqvGGRhR/57gMv+TJ1wjWs
        LcaF8DuFfNo8XO65iPo6f0JL23zf+mBV4cbSxf1Z4F4KCzKANAJW6dE0KfH0Np1k
        UDnPBxj4/9s+SLSYTaUVIeff6wu+ygIqm+/o2CEaucUVXqflM8jk
        -----END RSA PRIVATE KEY-----
    ssl:
      skip_cert_verify: true
    stager:
      cc:
        base_url: https://api.9.91.39.29.xip.io
        basic_auth_password: internal-password
        external_port: 9022
        staging_upload_password: upload-password
        staging_upload_user: upload-user
      diego_api_url: http://:@receptor.service.consul:8888
      docker_registry_url: http://9.91.39.37:8080
      log_level: null
    tps:
      cc:
        base_url: https://api.9.91.39.29.xip.io
        basic_auth_password: internal-password
        external_port: 9022
        staging_upload_password: upload-password
        staging_upload_user: upload-user
      diego_api_url: http://:@receptor.service.consul:8888
      log_level: null
  etcd:
    machines:
    - 10.10.10.143
  loggregator_endpoint:
    shared_secret: loggregator-secret
  nats:
    machines:
    - 10.10.10.142
    password: nats
    port: 4222
    user: nats
  syslog_daemon_config:
    address: null
    port: null
releases:
- name: diego
  version: latest
- name: cf
  version: latest
resource_pools:
- cloud_properties:
    instance_type: m1.medium
  name: brain_z1
  network: diego1
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: brain_z2
  network: diego2
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: brain_z3
  network: diego3
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: cc_bridge_z1
  network: diego1
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: cc_bridge_z2
  network: diego2
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: cc_bridge_z3
  network: diego3
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: cell_z1
  network: diego1
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: cell_z2
  network: diego2
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: cell_z3
  network: diego3
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: etcd_z1
  network: diego1
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: etcd_z2
  network: diego2
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties:
    instance_type: m1.medium
  name: etcd_z3
  network: diego3
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties: 
    instance_type: m1.medium
  name: route_emitter_z1
  network: diego1
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties: 
    instance_type: m1.medium
  name: route_emitter_z2
  network: diego2
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
- cloud_properties: 
    instance_type: m1.medium
  name: route_emitter_z3
  network: diego3
  stemcell:
    name: bosh-openstack-kvm-ubuntu-trusty-go_agent-raw
    version: latest
update:
  canaries: 1
  canary_watch_time: 5000-120000
  max_in_flight: 1
  serial: false
  update_watch_time: 5000-120000
emalm commented 9 years ago

Hi, @guanglinlv,

We have a similar configuration for garden-linux in the diego-release testing pipeline, and that has correctly passed the automated tests we have that exercise our experimental internal caching registry. That registry is based on v0.9.1 of the python-based docker registry, and is configured to be insecure. Can you provide any more information about the registry you're running, or the URI of the docker image you're trying to run? Perhaps your registry is configured differently from ours, and the docker code that garden-linux uses may interpret its responses differently to send you down this particular pathway.

Thanks, Eric

emalm commented 9 years ago

Also, cc: @hsiliev and @georgethebeatle, who may have some additional insight, and @glyn and @julz from the garden-linux team.

julz commented 9 years ago

Hi @guanglinlv, I think you're running in to this story in the garden-linux backlog: https://www.pivotaltracker.com/story/show/91802212. If you're interested in submitting a PR it should be fixable within garden-linux by passing an empty (rather than nil) authConfig struct as the first argument to RegistryNewSession() on L49 of repository_provider.go. Alternatively we'll fix this as soon as we get to the story in our backlog. In the meantime please accept our apologies for the inconvenience and our thanks for reporting the bug!

guanglinlv commented 9 years ago

hi @ematpl ,

My prviate registry has a CA signed cert.actually,it was behind an nginx server which was terminating the SSL and forward request to registry:5000

I did little change on stager at L97 of docker_backend.go and getDockerRegistryServices, it let my diego work with my insecure private registry.

diff --git a/backend/docker_backend.go b/backend/docker_backend.go
index 9ee3ec6..7a2ed7e 100644
--- a/backend/docker_backend.go
+++ b/backend/docker_backend.go
@@ -93,7 +93,7 @@ func (backend *dockerBackend) BuildRecipe(stagingGuid string, request cc_message

        runActionArguments := []string{"-outputMetadataJSONFilename", DockerBuilderOutputPath, "-dockerRef", lifecycleData.DockerImageUrl}
        if backend.config.DockerRegistry != nil {
-               registryServices, err := getDockerRegistryServices(backend.config.ConsulCluster)
+               registryServices, err := getDockerRegistryServices(backend.config.ConsulCluster,backend.config.DockerRegistry)
                if err != nil {
                        return receptor.TaskCreateRequest{}, err
                }
@@ -270,7 +270,7 @@ func buildDockerRegistryAddresses(services []consulServiceInfo) []string {
        return registries
 }

-func getDockerRegistryServices(consulCluster string) ([]consulServiceInfo, error) {
+func getDockerRegistryServices(consulCluster string,dockerRegistry *DockerRegistry) ([]consulServiceInfo, error) {
        response, err := http.Get(consulCluster + "/v1/catalog/service/docker-registry")
        if err != nil {
                return nil, err
@@ -288,6 +288,14 @@ func getDockerRegistryServices(consulCluster string) ([]consulServiceInfo, error
                return nil, err
        }

+       //hacked,get private docker registry from docker_registry_url
+       parts, err := url.Parse(dockerRegistry.URL)
+       if err != nil {
+               return nil,err
+       }
+       registry_host := strings.Split(parts.Host,":")[0]
+       ips = []consulServiceInfo{{Address: registry_host}}
+
        if len(ips) == 0 {
                return nil, ErrMissingDockerRegistry
        }

Little change on L51 of session.go help me to avoid the crash problem.

diff --git a/Godeps/_workspace/src/github.com/docker/docker/registry/session.go b/Godeps/_workspace/src/github.com/docker/docker/registry/session.go
index ba6df35..d6c5dfb 100644
--- a/Godeps/_workspace/src/github.com/docker/docker/registry/session.go
+++ b/Godeps/_workspace/src/github.com/docker/docker/registry/session.go
@@ -48,7 +48,7 @@ func NewSession(authConfig *AuthConfig, factory *utils.HTTPRequestFactory, endpo

        // If we're working with a standalone private registry over HTTPS, send Basic Auth headers
        // alongside our requests.
-       if r.indexEndpoint.VersionString(1) != IndexServerAddress() && r.indexEndpoint.URL.Scheme == "https" {
+       if r.indexEndpoint.VersionString(1) != IndexServerAddress() && r.indexEndpoint.URL.Scheme == "https" && r.indexEndpoint.secure {
                info, err := r.indexEndpoint.Ping()
                if err != nil {
                        return nil, err

@julz,thanks for your jumping,yes it's the same as issue, my prviate registry has a CA signed cert.

thanks all.

emalm commented 9 years ago

Closing this since this is a known issue with garden-linux, with a corresponding, prioritized story in the Garden backlog.