swipely / iam-docker

Use different IAM roles for each Docker container on an EC2 instance
MIT License
211 stars 22 forks source link

Assuming roles won't work with custom network interfaces #6

Closed patagona-afriemann closed 8 years ago

patagona-afriemann commented 8 years ago

Running containers with custom networks will result in the IPAddress field being empty

$ docker run --net local -l com.swipely.iam-docker.iam-profile=arn:aws:iam::************:role/some-role tutum/curl bash -c 'while true; do sleep 2; done' 
$ docker inspect bd46f10c6b87
[
        ....,
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "a46ba1358bd0d7096e7f054cbaf6162fb2353ac519f324417fab6d567b6bf65b",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/a46ba1358bd0",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "local": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "cf66e08b22506bf00ccc46dae42df46531628b7413a058c202983ebcff4d7957",
                    "EndpointID": "4c57b930b76089ce5ef655b1561cc283287c2d16459fb4a29adbc4050a914e21",
                    "Gateway": "192.168.0.1",
                    "IPAddress": "192.168.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:c0:a8:00:02"
                }
            }
        }
    }
]

which in turn will result in iam-docker not being able to find the container

2016-07-06T18:34:09Z [docker] Adding container event-handler=2 id=bd46f10c6b87784440b5cfb1bd34445292e5b6d7bcfea05be36fa28b30d5761e event=start
2016-07-06T18:34:09Z [docker] Fetching credentials event-handler=2 id=bd46f10c6b87784440b5cfb1bd34445292e5b6d7bcfea05be36fa28b30d5761e event=start role=arn:aws:iam::************:role/some-role
2016-07-06T18:20:56Z [iam] Credential successfully refreshed arn=arn:aws:iam::************:role/some-role
2016-07-06T18:38:00Z [http] Unable to find credentials method=GET remoteAddr=192.168.0.2:49782 error="Unable to find container for IP: 192.168.0.2" path=/latest/meta-data/iam/security-credentials/

Debug output shows the IP is clearly not set when creating the config in docker/container_store.go, line 171. As stated in https://github.com/docker/docker/issues/21658 the toplevel IPAddress will only be set for the default bridge device.

Our current use case is a docker swarm. This makes it practically impossible to use iam-docker with docker-compose files and overlay networks.

I propose an additional label to set the network interface and request the IPAddress from the specified interface instead of the toplevel IPAddress field, probably with a fallback to "bridge".

p.s. I might be able to supply a patch until next week, however I never worked with go and would be happy to accept another solution instead.

nahiluhmot commented 8 years ago

Thanks for opening this issue @patagona-afriemann! I wasn't aware of Docker not setting the top level IPAddress field. Patching this should be simple enough, I'll open a pull request and tag you on it so that you can test it out.

nahiluhmot commented 8 years ago

Pull request: https://github.com/swipely/iam-docker/pull/7

nahiluhmot commented 8 years ago

Merged #7 yesterday and validated it locally. Going to close this issue -- feel free to reopen if it's not resolved.