runsc doesn't work with user defined bridge network

google / gvisor

Application Kernel for Containers

https://gvisor.dev

Apache License 2.0

15.85k stars 1.3k forks source link

runsc doesn't work with user defined bridge network #768

Closed Anjali05 closed 5 years ago

Anjali05 commented 5 years ago

I am trying to run a workload: https://github.com/parsa-epfl/cloudsuite/blob/master/docs/benchmarks/data-caching.md. It uses a user-defined bridge network, although the container runs fine with runc, it fails with runsc giving a DNS error on the client-side. All the other workloads in this suite is also failing on runsc as most of them have user defined bridge. I tried running normal Ubuntu container with user-defined network and then ran apt update but it's not able to connect to the archive site. I am wondering if there is any other network setting I need to define in runsc if I am using user-defined bridge network, the goal is to make two containers communicate with each other on the same host. I am using --net flag with docker run to specify the bridge.

fvoznika commented 5 years ago

When using user defined bridge, docker sets up a embedded DNS server bound to the loopback interface on address 127.0.0.10. runsc network is isolated from the host and the loopback interface is contained inside the sandbox, therefore it cannot access the DNS server setup by Docker (on the host network). You can workaround it by using the --link option to connect containers.

Anjali05 commented 5 years ago

@fvoznika Is there any plan of enabling network communication between the containers running with runsc using netstack?

ianlewis commented 5 years ago

I think maybe we could add something to the docs to document the workaround.

fvoznika commented 5 years ago

--link doesn't actually work with user defined bridge because Docker adds the alias to the embedded DNS instead of changing /etc/hosts file like it does for the default bridge. So in terms of workarounds, there are a few options:

Use default bridge network with --link to connect containers.
Use --network=host in runsc, however beware that it will use the host network stack and is less secure.
Use IPs instead of container names.
Use Kubernetes. Container name lookup works fine in Kubernetes.

@Anjali05 there are no plans currently to support Docker's embedded DNS as this breaks the sandbox isolation. Note that technically containers can communicate with each other, it's just the name lookup that doesn't work in Docker's user defined bridge.

@ianlewis, sure thing. I'll add it to the docs.

Anjali05 commented 5 years ago

@fvoznika Thanks! I was using internal IP which is working.

fvoznika commented 5 years ago

Added workarounds to the FAQ.