search

kata-containers / runtime

Kata Containers version 1.x runtime (for version 2.x see https://github.com/kata-containers/kata-containers).
https://katacontainers.io/
Apache License 2.0
2.1k stars 375 forks source link

Bug: Message too long 13110 #386

Closed linxiulei closed 5 years ago

linxiulei commented 6 years ago

Description of problem

Create a container with an env file, which has size bigger than 10240 bytes, cmd like

docker run --runtime cc-runtime --env-file /tmp/env -it busybox sh

then

docker run --runtime cc-runtime busybox ls

it has no response anymore

Expected result

it's supposed to run normally

Actual result

docker cmd no response

Meta details

Running cc-collect-data.sh version 3.0.22 (commit 4b569ac) at 2018-06-12.17:13:02.305371930+0800.

Runtime is /usr/bin/cc-runtime.

cc-env

Output of "/usr/bin/cc-runtime cc-env":

[Meta]
  Version = "1.0.9"

[Runtime]
  Debug = false
  [Runtime.Version]
    Semver = "3.0.22"
    Commit = "4b569ac"
    OCI = "1.0.1"
  [Runtime.Config]
    Path = "/etc/clear-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  Version = "QEMU emulator version 2.7.1 (), Copyright (c) 2003-2016 Fabrice Bellard and the QEMU Project developers"
  Path = "/usr/local/bin/qemu-system-x86_64"
  Debug = false
  BlockDeviceDriver = "virtio-scsi"

[Image]
  Path = "/usr/share/clear-containers/cc-20640-agent-62f3f7.img"

[Kernel]
  Path = "/usr/share/clear-containers/vmlinuz-4.9.47-82.1.alios7.container"
  Parameters = ""

[Proxy]
  Type = "ccProxy"
  Version = "Version: 3.0.22+git.e7ee393-9bf9e916e8a6bc173fa71fe1fe796b37d077d62f"
  Path = "/usr/libexec/clear-containers/cc-proxy"
  Debug = false

[Shim]
  Type = "ccShim"
  Version = "shim version: 3.0.22 (commit: 10a255b)"
  Path = "/usr/libexec/clear-containers/cc-shim"
  Debug = false

[Agent]
  Type = "hyperstart"
  Version = "<<unknown>>"

[Host]
  Kernel = "4.4.95-1.al7.x86_64"
  Architecture = "amd64"
  VMContainerCapable = true
  [Host.Distro]
    Name = "Alibaba Cloud Enterprise Linux Server"
    Version = "17.01"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Xeon(R) CPU E3-1240 v6 @ 3.70GHz"

Runtime config files

Runtime default config files

/etc/clear-containers/configuration.toml
/usr/share/defaults/clear-containers/configuration.toml

Runtime config file contents

Output of "cat "/etc/clear-containers/configuration.toml"":

[hypervisor.qemu]
path = "/usr/local/bin/qemu-system-x86_64"
kernel = "/usr/share/clear-containers/vmlinuz.container"
image = "/usr/share/clear-containers/clear-containers.img"
machine_type = "pc"
kernel_params = ""
firmware = ""
machine_accelerators=""
default_vcpus = 1
default_bridges = 1
disable_block_device_use = true
#block_device_driver = "virtio-scsi"

[proxy.cc]
path = "/usr/libexec/clear-containers/cc-proxy"

[shim.cc]
path = "/usr/libexec/clear-containers/cc-shim"

[agent.cc]

[runtime]
internetworking_model="macvtap"

Output of "cat "/usr/share/defaults/clear-containers/configuration.toml"":

# XXX: WARNING: this file is auto-generated.
# XXX:
# XXX: Source file: "config/configuration.toml.in"
# XXX: Project:
# XXX:   Name: Intel® Clear Containers
# XXX:   Type: cc

[hypervisor.qemu]
path = "/usr/bin/qemu-system-x86_64"
kernel = "/usr/share/clear-containers/vmlinuz.container"
image = "/usr/share/clear-containers/clear-containers.img"
machine_type = "pc"

# Optional space-separated list of options to pass to the guest kernel.
# For example, use `kernel_params = "vsyscall=emulate"` if you are having
# trouble running pre-2.15 glibc.
#
# WARNING: - any parameter specified here will take priority over the default
# parameter value of the same name used to start the virtual machine.
# Do not set values here unless you understand the impact of doing so as you
# may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a
# container and look for 'default-kernel-parameters' log entries.
kernel_params = ""

# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = ""

# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
machine_accelerators=""

# Default number of vCPUs per POD/VM:
# unspecified or 0                --> will be set to 1
# < 0                             --> will be set to the actual number of physical cores
# > 0 <= number of physical cores --> will be set to the specified number
# > number of physical cores      --> will be set to the actual number of physical cores
default_vcpus = 1

# Bridges can be used to hot plug devices.
# Limitations:
# * Currently only pci bridges are supported
# * Until 30 devices per bridge can be hot plugged.
# * Until 5 PCI bridges can be cold plugged per VM.
#   This limitation could be a bug in qemu or in the kernel
# Default number of bridges per POD/VM:
# unspecified or 0   --> will be set to 1
# > 1 <= 5           --> will be set to the specified number
# > 5                --> will be set to 5
default_bridges = 1

# Default memory size in MiB for POD/VM.
# If unspecified then it will be set 2048 MiB.
#default_memory = 2048

# Disable block device from being used for a container's rootfs.
# In case of a storage driver like devicemapper where a container's 
# root file system is backed by a block device, the block device is passed
# directly to the hypervisor for performance reasons. 
# This flag prevents the block device from being passed to the hypervisor, 
# 9pfs is used instead to pass the rootfs.
disable_block_device_use = false

# Block storage driver to be used for the hypervisor in case the container
# rootfs is backed by a block device. This is either virtio-scsi or 
# virtio-blk.
block_device_driver = "virtio-scsi"

# Enable pre allocation of VM RAM, default false
# Enabling this will result in lower container density
# as all of the memory will be allocated and locked
# This is useful when you want to reserve all the memory
# upfront or in the cases where you want memory latencies
# to be very predictable
# Default false
#enable_mem_prealloc = true

# Enable huge pages for VM RAM, default false
# Enabling this will result in the VM memory
# being allocated using huge pages.
# This is useful when you want to use vhost-user network
# stacks within the container. This will automatically 
# result in memory pre allocation
#enable_hugepages = true

# Enable swap of vm memory. Default false.
# The behaviour is undefined if mem_prealloc is also set to true
#enable_swap = true

# This option changes the default hypervisor and kernel parameters
# to enable debug output where available. This extra output is added
# to the proxy logs, but only when proxy debug is also enabled.
# 
# Default false
#enable_debug = true

# Disable the customizations done in the runtime when it detects
# that it is running on top a VMM. This will result in the runtime
# behaving as it would when running on bare metal.
# 
#disable_nesting_checks = true

[proxy.cc]
path = "/usr/libexec/clear-containers/cc-proxy"

# If enabled, proxy messages will be sent to the system log
# (default: disabled)
#enable_debug = true

[shim.cc]
path = "/usr/libexec/clear-containers/cc-shim"

# If enabled, shim messages will be sent to the system log
# (default: disabled)
#enable_debug = true

[agent.cc]
# There is no field for this section. The goal is only to be able to
# specify which type of agent the user wants to use.

[runtime]
# If enabled, the runtime will log additional debug messages to the
# system log
# (default: disabled)
#enable_debug = true
#
# Internetworking model
# Determines how the VM should be connected to the
# the container network interface
# Options:
#
#   - bridged
#     Uses a linux bridge to interconnect the container interface to
#     the VM. Works for most cases except macvlan and ipvlan.
#
#   - macvtap
#     Used when the Container network interface can be bridged using
#     macvtap.
internetworking_model="macvtap"

Agent

version:

unknown

Logfiles

Runtime logs

Recent runtime problems found in system journal:

time="2018-06-05T17:35:50.249094397+08:00" level=error msg="Unable to launch qemu: exit status 127" arch=amd64 name=cc-runtime pid=32204 source=virtcontainers subsystem=qmp
time="2018-06-05T17:35:50.249113887+08:00" level=error msg="/usr/local/bin/qemu-system-x86_64: error while loading shared libraries: libpixman-1.so.0: cannot open shared object file: No such file or directory\n" arch=amd64 name=cc-runtime pid=32204 source=virtcontainers subsystem=qmp
time="2018-06-05T17:35:50.249133891+08:00" level=error msg="/usr/local/bin/qemu-system-x86_64: error while loading shared libraries: libpixman-1.so.0: cannot open shared object file: No such file or directory\n" command=create name=cc-runtime pid=32204 source=runtime
time="2018-06-05T17:35:50.502800927+08:00" level=error msg="Container ID (5f7856edecf256229462732c8ad10a781bdfa9b1d84f575a1361fb1a782f743b) does not exist" command=delete name=cc-runtime pid=32235 source=runtime
time="2018-06-05T17:35:50.522505345+08:00" level=error msg="Container ID (5f7856edecf256229462732c8ad10a781bdfa9b1d84f575a1361fb1a782f743b) does not exist" command=delete name=cc-runtime pid=32242 source=runtime
time="2018-06-05T17:36:59.573735002+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=32378 source=runtime
time="2018-06-05T17:36:59.636731468+08:00" level=error msg="Container ID (248856dd7fb118363b3756390c53557307eb02803757573e09c35c1545e52a4f) does not exist" command=delete name=cc-runtime pid=32393 source=runtime
time="2018-06-05T17:40:39.489639241+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=32666 source=runtime
time="2018-06-05T17:40:39.556675805+08:00" level=error msg="Container ID (f6700aacfefc8a738e67f3dfce0b1d0c2db5240f43edd91c3b6da56175de4a12) does not exist" command=delete name=cc-runtime pid=32681 source=runtime
time="2018-06-11T21:31:31.857599961+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=3285 source=runtime
time="2018-06-11T21:31:31.913060602+08:00" level=error msg="Container ID (6cbf4234cd33e35a005d75ae0c1e4716e3729f50f84d344fbfee2c400e3dd009) does not exist" command=delete name=cc-runtime pid=3299 source=runtime
time="2018-06-11T21:35:52.837583706+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=3401 source=runtime
time="2018-06-11T21:35:52.897008472+08:00" level=error msg="Container ID (ae6e4935a0ea326d7fe0c9b2a071323674638c663bdc25262a77f618dc15f024) does not exist" command=delete name=cc-runtime pid=3415 source=runtime
time="2018-06-11T21:36:07.301590383+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=3513 source=runtime
time="2018-06-11T21:36:07.356784493+08:00" level=error msg="Container ID (76b299724ada517f781035cd730e86162f59624cfc2b603f246b01a09d2c6766) does not exist" command=delete name=cc-runtime pid=3527 source=runtime
time="2018-06-12T14:55:10.20973936+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=3758 source=runtime
time="2018-06-12T14:55:10.264820273+08:00" level=error msg="Container ID (c1c2b0d433f6d84614a3ae5984f066558660b7dddff7e35b8611965e612e1c01) does not exist" command=delete name=cc-runtime pid=3772 source=runtime
time="2018-06-12T15:19:28.593691271+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=4179 source=runtime
time="2018-06-12T15:19:28.652752153+08:00" level=error msg="Container ID (5f896a89e11f6041b34d4d40d98f851759eadaf21e83b3148673e3fc75ba4e1d) does not exist" command=delete name=cc-runtime pid=4193 source=runtime
time="2018-06-12T15:21:58.885583319+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=4298 source=runtime
time="2018-06-12T15:21:58.961126395+08:00" level=error msg="Container ID (bcb9899ba725ae06aebf61c5ed892d02142a69803e39ed0f7bb609d8c28af673) does not exist" command=delete name=cc-runtime pid=4313 source=runtime
time="2018-06-12T15:22:20.717589638+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=4408 source=runtime
time="2018-06-12T15:22:20.789073883+08:00" level=error msg="Container ID (8d8919baeae3eb42e369b90715b6e135e4fca38443dcaa3783e8ecebfca7214f) does not exist" command=delete name=cc-runtime pid=4422 source=runtime
time="2018-06-12T15:22:50.349698523+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=4522 source=runtime
time="2018-06-12T15:22:50.417019284+08:00" level=error msg="Container ID (6d20e7c8d09d4f3ab2410afe7e50b64e2b20d95da534c92c4e19a8aec0d64b2f) does not exist" command=delete name=cc-runtime pid=4536 source=runtime
time="2018-06-12T15:30:45.898357273+08:00" level=error msg="Failed to start container" arch=amd64 container-id=a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861 error="message too long 15306" name=cc-runtime pid=4695 pod-id=a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861 source=virtcontainers subsystem=container
time="2018-06-12T15:30:56.005170021+08:00" level=error msg="message too long 15306" command=start name=cc-runtime pid=4695 source=runtime
time="2018-06-12T15:30:56.006973934+08:00" level=error msg="Container a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861 not ready or running, cannot send a signal" command=kill name=cc-runtime pid=4702 source=runtime
time="2018-06-12T16:01:11.100008447+08:00" level=error msg=EOF command=delete name=cc-runtime pid=4709 source=runtime
time="2018-06-12T16:01:12.105307411+08:00" level=warning msg="Retrying proxy connection" arch=amd64 attempt=12 error="dial unix /run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock: connect: connection refused" name=cc-runtime pid=5616 proxy-address=/run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock proxy-network=unix remaining-time-secs=4.00 source=virtcontainers subsystem=hyper
time="2018-06-12T16:01:13.106303355+08:00" level=warning msg="Retrying proxy connection" arch=amd64 attempt=22 error="dial unix /run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock: connect: connection refused" name=cc-runtime pid=5616 proxy-address=/run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock proxy-network=unix remaining-time-secs=3.00 source=virtcontainers subsystem=hyper
time="2018-06-12T16:01:14.107217997+08:00" level=warning msg="Retrying proxy connection" arch=amd64 attempt=32 error="dial unix /run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock: connect: connection refused" name=cc-runtime pid=5616 proxy-address=/run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock proxy-network=unix remaining-time-secs=2.00 source=virtcontainers subsystem=hyper
time="2018-06-12T16:01:15.108201533+08:00" level=warning msg="Retrying proxy connection" arch=amd64 attempt=42 error="dial unix /run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock: connect: connection refused" name=cc-runtime pid=5616 proxy-address=/run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock proxy-network=unix remaining-time-secs=1.00 source=virtcontainers subsystem=hyper
time="2018-06-12T16:01:16.109173635+08:00" level=error msg="failed to connect to proxy after 5s: dial unix /run/virtcontainers/pods/a966e6db6dffda24cb987d32d634d07710592b9225d43333529aa3421f1f4861/proxy.sock: connect: connection refused" command=delete name=cc-runtime pid=5616 source=runtime
time="2018-06-12T16:04:36.089568853+08:00" level=error msg="Container not ready or running, impossible to signal the container" command=kill name=cc-runtime pid=6437 source=runtime
time="2018-06-12T16:04:36.154565655+08:00" level=error msg="Container ID (e4f57e90282b71360f44cdcbfc9dac5e356f4bedb46f33297bb1021e2d2afe30) does not exist" command=delete name=cc-runtime pid=6451 source=runtime
time="2018-06-12T16:35:27.976301047+08:00" level=error msg="Failed to start container" arch=amd64 container-id=17ac9b35b275ee43460df282ae066a760922dd1f4e39f90c7571910988b18e4a error="message too long 13361" name=cc-runtime pid=6970 pod-id=17ac9b35b275ee43460df282ae066a760922dd1f4e39f90c7571910988b18e4a source=virtcontainers subsystem=container
time="2018-06-12T16:35:38.083085423+08:00" level=error msg="message too long 13361" command=start name=cc-runtime pid=6970 source=runtime
time="2018-06-12T16:35:38.086313173+08:00" level=error msg="Container 17ac9b35b275ee43460df282ae066a760922dd1f4e39f90c7571910988b18e4a not ready or running, cannot send a signal" command=kill name=cc-runtime pid=6977 source=runtime

Proxy logs

Recent proxy problems found in system journal:

time="2018-06-05T17:36:59.504481432+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/248856dd7fb118363b3756390c53557307eb02803757573e09c35c1545e52a4f/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=32359 source=proxy
time="2018-06-05T17:40:39.417955895+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/f6700aacfefc8a738e67f3dfce0b1d0c2db5240f43edd91c3b6da56175de4a12/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=32651 source=proxy
time="2018-06-11T21:31:31.786326778+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/6cbf4234cd33e35a005d75ae0c1e4716e3729f50f84d344fbfee2c400e3dd009/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=3270 source=proxy
time="2018-06-11T21:35:52.765297928+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/ae6e4935a0ea326d7fe0c9b2a071323674638c663bdc25262a77f618dc15f024/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=3386 source=proxy
time="2018-06-11T21:36:07.225444499+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/76b299724ada517f781035cd730e86162f59624cfc2b603f246b01a09d2c6766/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=3498 source=proxy
time="2018-06-12T14:55:10.131579257+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/c1c2b0d433f6d84614a3ae5984f066558660b7dddff7e35b8611965e612e1c01/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=3743 source=proxy
time="2018-06-12T15:19:28.525290347+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/5f896a89e11f6041b34d4d40d98f851759eadaf21e83b3148673e3fc75ba4e1d/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=4164 source=proxy
time="2018-06-12T15:21:58.818998363+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/bcb9899ba725ae06aebf61c5ed892d02142a69803e39ed0f7bb609d8c28af673/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=4284 source=proxy
time="2018-06-12T15:22:20.650897921+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/8d8919baeae3eb42e369b90715b6e135e4fca38443dcaa3783e8ecebfca7214f/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=4393 source=proxy
time="2018-06-12T15:22:50.284018176+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/6d20e7c8d09d4f3ab2410afe7e50b64e2b20d95da534c92c4e19a8aec0d64b2f/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=4507 source=proxy
time="2018-06-12T16:04:36.013287766+08:00" level=error msg="error serving client: write unix /run/virtcontainers/pods/e4f57e90282b71360f44cdcbfc9dac5e356f4bedb46f33297bb1021e2d2afe30/proxy.sock->@: use of closed network connection" client=4 name=cc-proxy pid=6419 source=proxy

Shim logs

No recent shim problems found in system journal.

Container manager details

Have docker

Docker

Output of "docker version":

Client:
 Version:      17.06.2-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 19:59:06 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.2-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 20:00:25 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of "docker info":

Containers: 37
 Running: 0
 Paused: 0
 Stopped: 37
Images: 5
Server Version: 17.06.2-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 107
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: cc-runtime runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 6e23458c129b551d5c9871e5174f6b1b7f6d1170
runc version: 810190ceaa507aa2727d7ae6f4790c76ec150bd2
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.95-1.al7.x86_64
Operating System: Alibaba Cloud Enterprise Linux Server 17.01 (Golden Toad)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.23GiB
Name: iz2zedl2kxe9e86xi3egfkz
ID: EPN6:NP54:DTXY:WMU7:SZPJ:C2NK:HSJR:WBXN:2C64:EAXI:CZ5C:CRHX
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

Output of "systemctl show docker":

Type=notify
Restart=on-failure
NotifyAccess=main
RestartUSec=100ms
TimeoutStartUSec=0
TimeoutStopUSec=1min 30s
WatchdogUSec=0
WatchdogTimestamp=Tue 2018-06-12 16:50:34 CST
WatchdogTimestampMonotonic=606048675098
StartLimitInterval=60000000
StartLimitBurst=3
StartLimitAction=none
FailureAction=none
PermissionsStartOnly=no
RootDirectoryStartOnly=no
RemainAfterExit=no
GuessMainPID=yes
MainPID=7724
ControlPID=0
FileDescriptorStoreMax=0
StatusErrno=0
Result=success
ExecMainStartTimestamp=Tue 2018-06-12 16:50:33 CST
ExecMainStartTimestampMonotonic=606047380043
ExecMainExitTimestampMonotonic=0
ExecMainPID=7724
ExecMainCode=0
ExecMainStatus=0
ExecStart={ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd ; ignore_errors=no ; start_time=[Tue 2018-06-12 16:50:33 CST] ; stop_time=[n/a] ; pid=7724 ; code=(null) ; status=0/0 }
ExecReload={ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
Slice=system.slice
ControlGroup=/system.slice/docker.service
MemoryCurrent=50569216
Delegate=yes
CPUAccounting=no
CPUShares=18446744073709551615
StartupCPUShares=18446744073709551615
CPUQuotaPerSecUSec=infinity
BlockIOAccounting=no
BlockIOWeight=18446744073709551615
StartupBlockIOWeight=18446744073709551615
MemoryAccounting=no
MemoryLimit=18446744073709551615
DevicePolicy=auto
UMask=0022
LimitCPU=18446744073709551615
LimitFSIZE=18446744073709551615
LimitDATA=18446744073709551615
LimitSTACK=18446744073709551615
LimitCORE=18446744073709551615
LimitRSS=18446744073709551615
LimitNOFILE=18446744073709551615
LimitAS=18446744073709551615
LimitNPROC=18446744073709551615
LimitMEMLOCK=65536
LimitLOCKS=18446744073709551615
LimitSIGPENDING=127826
LimitMSGQUEUE=819200
LimitNICE=0
LimitRTPRIO=0
LimitRTTIME=18446744073709551615
OOMScoreAdjust=0
Nice=0
IOScheduling=0
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SecureBits=0
CapabilityBoundingSet=18446744073709551615
MountFlags=0
PrivateTmp=no
PrivateNetwork=no
PrivateDevices=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=no
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=0
RuntimeDirectoryMode=0755
KillMode=process
KillSignal=15
SendSIGKILL=yes
SendSIGHUP=no
Id=docker.service
Names=docker.service
Requires=basic.target
Wants=network-online.target system.slice
Conflicts=shutdown.target
Before=shutdown.target
After=basic.target systemd-journald.socket system.slice firewalld.service network-online.target
Documentation=https://docs.docker.com
Description=Docker Application Container Engine
LoadState=loaded
ActiveState=active
SubState=running
FragmentPath=/usr/lib/systemd/system/docker.service
UnitFileState=disabled
UnitFilePreset=disabled
InactiveExitTimestamp=Tue 2018-06-12 16:50:33 CST
InactiveExitTimestampMonotonic=606047380061
ActiveEnterTimestamp=Tue 2018-06-12 16:50:34 CST
ActiveEnterTimestampMonotonic=606048675126
ActiveExitTimestamp=Tue 2018-06-12 16:50:18 CST
ActiveExitTimestampMonotonic=606032327583
InactiveEnterTimestamp=Tue 2018-06-12 16:50:33 CST
InactiveEnterTimestampMonotonic=606047366630
CanStart=yes
CanStop=yes
CanReload=yes
CanIsolate=no
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=no
IgnoreOnSnapshot=no
NeedDaemonReload=no
JobTimeoutUSec=0
JobTimeoutAction=none
ConditionResult=yes
AssertResult=yes
ConditionTimestamp=Tue 2018-06-12 16:50:33 CST
ConditionTimestampMonotonic=606047366796
AssertTimestamp=Tue 2018-06-12 16:50:33 CST
AssertTimestampMonotonic=606047366796
Transient=no

No kubectl

Packages

No dpkg Have rpm Output of "rpm -qa|egrep "(cc-oci-runtime|cc-proxy|cc-runtime|cc-shim|kata-proxy|kata-runtime|kata-shim|clear-containers-image|linux-container|qemu-lite|qemu-system-x86)"":

clear-containers-image-20640-52.1.alios7.x86_64
cc-runtime-bin-3.0.22+git.4b569ac-1.26.7.x86_64
cc-shim-bin-3.0.22+git.10a255b-27.1.alios7.x86_64
cc-proxy-bin-3.0.22+git.e7ee393-30.1.alios7.x86_64
cc-proxy-3.0.22+git.e7ee393-30.1.alios7.x86_64
linux-container-4.9.47-82.1.alios7.x86_64
cc-runtime-config-3.0.22+git.4b569ac-1.26.7.x86_64

Note1: I was using ClearContainer-3.0, however it is reproducible with kata

Note2: Actually it involves two bugs

1) https://github.com/containerd/containerd/pull/872

it's been fixed

2) https://github.com/kata-containers/runtime/blob/ca9f7abba96e5c4db734673b9e7d870076d715e2/virtcontainers/pkg/hyperstart/hyperstart.go#L394

Here it checks length if bigger than 10240 , which is no longer needed (pls correct me if I am wrong). Agent that receives message at other side have no length limit if it is smaller than UInt32

egernst commented 6 years ago

@malc0lm, @linxiulei - can you provide details to reproduce on Kata? We don't use hyper start in Kata. It should be specific to the new gRPC protocol / agent.

grahamwhaley commented 6 years ago

I had a feeling we had a test for this, but I cannot locate it. @GabyCT @devimc - any memories of this? If we don't have an existing test, can we add one with this PR maybe to check the fix is working? :-)

sboeuf commented 6 years ago

This should not be posted on this repo since the issue is related to Clear Containers (using cc-runtime in the Docker command). That being said, if this is a Clear Containers issue, you should switch to Kata Containers to get some support when running into issues.

caoruidong commented 5 years ago

Close this since it is not related to kata.