Unable to load balance a service that uses net: host when roles: - global is set

[derekdon]

This might not be an issue and is perhaps just a lack of docker networking understanding... but I'm unable to load balance a service that uses net: host when the load balancer service is given roles: - global. The load balancer works as expected with roles global in place for all services that use net: bridge, but is only able to resolve services that use net: host when the roles: - global flag is removed. Any ideas what's happening here?

Doesn't work:

web:
  deployment_strategy: high_availability
  target_num_containers: 3
  image: 'someimage:latest'
  net: host
  ports:
    - '4984:4984'
    - '4985:4985'
  restart: on-failure
  tags:
    - web
  volumes:
    - '/tmp:/tmp/config'
loadbalancer:
  deployment_strategy: high_availability
  target_num_containers: 1
  image: 'dockercloud/haproxy:1.6.0'
  links:
    - web
  net: bridge
  ports:
    - '80:80'
    - '443:443'
  restart: always
  roles:
    - global
  tags:
    - loadbalancer

Works with global role removed:

web:
  deployment_strategy: high_availability
  target_num_containers: 3
  image: 'someimage:latest'
  net: host
  ports:
    - '4984:4984'
    - '4985:4985'
  restart: on-failure
  tags:
    - web
  volumes:
    - '/tmp:/tmp/config'
loadbalancer:
  deployment_strategy: high_availability
  target_num_containers: 1
  image: 'dockercloud/haproxy:1.6.0'
  links:
    - web
  net: bridge
  ports:
    - '80:80'
    - '443:443'
  restart: always
  tags:
    - loadbalancer

Works with global role present but linked service is using net: bridge:

web:
  deployment_strategy: high_availability
  target_num_containers: 3
  image: 'someimage:latest'
  net: bridge
  ports:
    - '4984:4984'
    - '4985:4985'
  restart: on-failure
  tags:
    - web
  volumes:
    - '/tmp:/tmp/config'
loadbalancer:
  deployment_strategy: high_availability
  target_num_containers: 1
  image: 'dockercloud/haproxy:1.6.0'
  links:
    - web
  net: bridge
  ports:
    - '80:80'
    - '443:443'
  restart: always
  roles:
    - global
  tags:
    - loadbalancer

Just to point out that two containers share the web node and they communicate on localhost so net: host is being used.

[tifayuki]

@derekdon When you setroles: globalon haproxy container, the only difference is that the you have a token added to the environment variable that allows the container to query dockercloud api. It has nothing to do with the net mode.

BTW, the net: bridge here is not as the same as bridge in docker. It is actually a overlay network using weave, which allows you to access containers over different nodes. I think it could be the network issue when you use net: host. We didn't test the host net mode on this image.

[derekdon]

@tifayuki Thanks for your speedy reply. Yeah that's what I found strange. Like you said and according to the docs a simple token is set on the container when roles global is set, but I suspect something else happens. It seems odd that by adding and removing this it breaks and fixes the resolve to the linked service.

We didn't test the host net mode on this image.

Thanks for the heads up... and the info about bridge.