search

google / gvisor

Application Kernel for Containers
https://gvisor.dev
Apache License 2.0
15.81k stars 1.3k forks source link

How to use network passthrough？ #86

Closed zhang2639 closed 6 years ago

zhang2639 commented 6 years ago

docker version 

Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.9.2
 Git commit:   9ee9f40
 Built:        Thu Apr 26 07:12:25 2018
 OS/Arch:      linux/amd64
 Experimental: false
 Orchestrator: swarm
Server:
 Engine:
  Version:      18.03.1-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   9ee9f40
  Built:        Thu Apr 26 07:23:03 2018
  OS/Arch:      linux/amd64
  Experimental: false

uname -a

Linux izuf65nf8wcjt73srsaudaz 4.9.79-009+ #3 SMP Thu Jul 5 14:26:02 CST 2018 x86_64 x86_64 x86_64 GNU/Linux

Full docker command you ran

docker run --runtime=runsc  -it --name test_pt  --cpuset-cpus="10,11,12,13" -m 2G netperf:2.7.0 /bin/bash

/etc/docker/daemon.json

{
    "runtimes": {
        "runsc": {
            "path": "/usr/local/bin/runsc",
            "runtimeArgs": [
                "--debug-log-dir=/tmp/runsc",
                "--debug",
                "--strace",
                "--network=host"
            ]
       }
    }
}

create log

I0705 19:59:37.896435   86483 x:0] ***************************
I0705 19:59:37.896492   86483 x:0] Args: [/usr/local/bin/runsc --debug-log-dir=/tmp/runsc --debug --strace --network=host --root /var/run/docker/runtime-runsc/moby --log /run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/log.json --log-format json create --bundle /var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a --pid-file /run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/init.pid --console-socket /tmp/pty199869016/pty.sock d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a]
I0705 19:59:37.896516   86483 x:0] PID: 86483
I0705 19:59:37.896527   86483 x:0] UID: 0, GID: 0
I0705 19:59:37.896533   86483 x:0] Configuration:
I0705 19:59:37.896542   86483 x:0]              RootDir: /var/run/docker/runtime-runsc/moby
I0705 19:59:37.896548   86483 x:0]              Platform: ptrace
I0705 19:59:37.896556   86483 x:0]              FileAccess: proxy, overlay: false
I0705 19:59:37.896564   86483 x:0]              Network: host, logging: false
I0705 19:59:37.896572   86483 x:0]              Strace: true, max size: 1024, syscalls: []
I0705 19:59:37.896580   86483 x:0] ***************************
D0705 19:59:37.897527   86483 x:0] Spec: &{Version:1.0.1 Process:0xc4202d24e0 Root:0xc4202d8520 Hostname:d9e2e3b913dc Mounts:[{Destination:/proc Type:proc Source:proc Options:[nosuid noexec nodev]} {Destination:/dev Type:tmpfs Source:tmpfs Options:[nosuid strictatime mode=755 size=65536k]} {Destination:/dev/pts Type:devpts Source:devpts Options:[nosuid noexec newinstance ptmxmode=0666 mode=0620 gid=5]} {Destination:/sys Type:sysfs Source:sysfs Options:[nosuid noexec nodev ro]} {Destination:/sys/fs/cgroup Type:cgroup Source:cgroup Options:[ro nosuid noexec nodev]} {Destination:/dev/mqueue Type:mqueue Source:mqueue Options:[nosuid noexec nodev]} {Destination:/etc/resolv.conf Type:bind Source:/home/docker/containers/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/resolv.conf Options:[rbind rprivate]} {Destination:/etc/hostname Type:bind Source:/home/docker/containers/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/hostname Options:[rbind rprivate]} {Destination:/etc/hosts Type:bind Source:/home/docker/containers/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/hosts Options:[rbind rprivate]} {Destination:/dev/shm Type:bind Source:/home/docker/containers/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/mounts/shm Options:[rbind rprivate]}] Hooks:0xc420162550 Annotations:map[] Linux:0xc4202f6000 Solaris:<nil> Windows:<nil>}
D0705 19:59:37.897593   86483 x:0] Spec.Hooks: &{Prestart:[{Path:/proc/85695/exe Args:[libnetwork-setkey d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a a35515341e7ab89c9c213b1af781b8effdb86645e1d0304171374115757d4293] Env:[] Timeout:<nil>}] Poststart:[] Poststop:[]}
D0705 19:59:37.897612   86483 x:0] Spec.Linux: &{UIDMappings:[] GIDMappings:[] Sysctl:map[] Resources:0xc42013ef00 CgroupsPath:/docker/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a Namespaces:[{Type:mount Path:} {Type:network Path:} {Type:uts Path:} {Type:pid Path:} {Type:ipc Path:}] Devices:[] Seccomp:0xc42013ad00 RootfsPropagation: MaskedPaths:[/proc/kcore /proc/keys /proc/latency_stats /proc/timer_list /proc/timer_stats /proc/sched_debug /proc/scsi /sys/firmware] ReadonlyPaths:[/proc/asound /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger] MountLabel: IntelRdt:<nil>}
D0705 19:59:37.897645   86483 x:0] Spec.Process: &{Terminal:true ConsoleSize:<nil> User:{UID:0 GID:0 AdditionalGids:[] Username:} Args:[/bin/bash] Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=d9e2e3b913dc TERM=xterm] Cwd:/ Capabilities:0xc4202a0200 Rlimits:[] NoNewPrivileges:false ApparmorProfile: OOMScoreAdj:0xc4202ec4d0 SelinuxLabel:}
D0705 19:59:37.897707   86483 x:0] Spec.Root: &{Path:/home/docker/overlay/f62132e1ddfa716fc104fd90862aa7fe370e0587fcf084d76d6a24e110651445/merged Readonly:false}
D0705 19:59:37.897726   86483 x:0] Create sandbox "d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a" in root dir: /var/run/docker/runtime-runsc/moby
D0705 19:59:37.897800   86483 x:0] Starting gofer: /usr/local/bin/runsc [--debug=true --debug-log-dir=/tmp/runsc --log=/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/log.json --log-format=json --network=host --root=/var/run/docker/runtime-runsc/moby --strace=true gofer --bundle /var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a --io-fds=3 --io-fds=4 --io-fds=5 --io-fds=6]
I0705 19:59:37.897999   86483 x:0] Gofer started, pid: 86489
I0705 19:59:37.898159   86483 x:0] Sandbox will be started in empty IPC and UTS namespaces
I0705 19:59:37.898170   86483 x:0] Sandbox will be started in the current PID namespace
I0705 19:59:37.898177   86483 x:0] Sandbox will be started in empty mount namespace
I0705 19:59:37.898184   86483 x:0] Sandbox will be started in the container's network namespace: {Type:network Path:}
I0705 19:59:37.898197   86483 x:0] Sandbox will be started in the current user namespace
D0705 19:59:37.898203   86483 x:0] Starting sandbox: /usr/local/bin/runsc [/usr/local/bin/runsc --debug=true --debug-log-dir=/tmp/runsc --log=/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a/log.json --log-format=json --network=host --root=/var/run/docker/runtime-runsc/moby --strace=true boot --bundle /var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a --controller-fd=3 --console=true --io-fds=4 --io-fds=5 --io-fds=6 --io-fds=7 --apply-caps=true]
I0705 19:59:37.899869   86483 x:0] Sandbox started, pid: 86494
D0705 19:59:37.899899   86483 x:0] Waiting for sandbox "d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a" creation
D0705 19:59:37.913294   86483 x:0] Save sandbox "d9e2e3b913dc5c618fff15e0721166ba384f20152d97ad287a8a381d256f376a"
I0705 19:59:37.915474   86483 x:0] Exiting with status: 0

In log, Network: host means I start container successfully. But I use ifconfig in container, it failed. See:

#ifconfig
SIOCGIFCONF: Inappropriate ioctl for device
eth0: error fetching interface information: Inappropriate ioctl for device

eth0 in host.

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.19.77.116  netmask 255.255.240.0  broadcast 172.19.79.255
        ether 00:16:3e:1c:30:51  txqueuelen 1000  (Ethernet)
        RX packets 639957  bytes 786758279 (750.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 405234  bytes 37599228 (35.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Is it because my network adapter doesn't fit ? So how to use network passthrough？ Thank you for helping me.

zhang2639 commented 6 years ago

hostname -I works.

majek commented 4 years ago

(a) passing --network=host to runsc means the userspace networking stack netstack won't be used. A "connect()" from within gvisor becomes a "connect()" on a host kernel.

(b) there is also --network=host that you need to pass on to the docker. which will keep your docker not to create a new dedicated network namespace.

Not sure if this helps.