VM factory broken in 1.4.0/1.3.1

[linxiulei]

Description of problem

kata-runtime factory init

check journal log by journalctl -t kata-runtime -f

time="2018-11-28T17:15:10.985796696+08:00" level=error msg="failed to create new vm" arch=amd64 command=factory error="Failed to check if grpc server is working: rpc error: code = Unavailable desc = transport is closing" name=kata-runtime pid=126348 source=virtcontainers vm=fe675d86-eceb-4de4-9557-cbe3d50802d2

I have seen kernel param 'systemd.unit=kata-containers.target' was not in qemu append, which causes failure to invoke grpc from kata-agent

[jodh-intel]

Hi @linxiulei - please could you provide further information on this issue?

[linxiulei]

Hi @linxiulei - please could you provide further information on this issue?

Randomly touch enter : )

[jodh-intel]

Hi @linxiulei - please can you:

• Run sudo kata-collect-data.sh.
• Check the output to ensure it doesn't contain any sensitive / personal information.
• Paste the output directly into this issue.

[linxiulei]

Meta details

Running kata-collect-data.sh version 1.4.0 (commit bf1cf68) at 2018-11-28.18:51:31.765059873+0800.

---

Runtime is /usr/bin/kata-runtime.

kata-env

Output of "/usr/bin/kata-runtime kata-env":

[Meta]
  Version = "1.0.19"

[Runtime]
  Debug = true
  DisableNewNetNs = false
  Path = "/usr/bin/kata-runtime"
  [Runtime.Version]
    Semver = "1.4.0"
    Commit = "bf1cf68"
    OCI = "1.0.1-dev"
  [Runtime.Config]
    Path = "/etc/kata-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  Version = "QEMU emulator version 2.11.0\nCopyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers"
  Path = "/usr/bin/qemu-lite-system-x86_64"
  BlockDeviceDriver = "virtio-scsi"
  EntropySource = "/dev/urandom"
  Msize9p = 8192
  MemorySlots = 10
  Debug = true
  UseVSock = false

[Image]
  Path = "/usr/share/kata-containers/kata-containers-image_clearlinux_1.4.0_agent_0aae82b.img"

[Kernel]
  Path = "/usr/share/kata-containers/vmlinuz-4.14.67-19.7.container"
  Parameters = "agent.log=debug"

[Initrd]
  Path = ""

[Proxy]
  Type = "kataProxy"
  Version = "kata-proxy version 1.4.0-209cb5f"
  Path = "/usr/libexec/kata-containers/kata-proxy"
  Debug = true

[Shim]
  Type = "kataShim"
  Version = "kata-shim version 1.4.0-147ffb1"
  Path = "/usr/libexec/kata-containers/kata-shim"
  Debug = true

[Agent]
  Type = "kata"

[Host]
  Kernel = "3.10.0-327.x86_64"
  Architecture = "amd64"
  VMContainerCapable = false
  SupportVSocks = false
  [Host.Distro]
    Name = "Linux Server"
    Version = "7.2"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz"

[Netmon]
  Version = "kata-netmon version 1.4.0"
  Path = "/usr/libexec/kata-containers/kata-netmon"
  Debug = false
  Enable = false

---

Runtime config files

Runtime default config files

/etc/kata-containers/configuration.toml
/usr/share/defaults/kata-containers/configuration.toml

Runtime config file contents

Output of "cat "/etc/kata-containers/configuration.toml"":

# Copyright (c) 2017-2018 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
#

# XXX: WARNING: this file is auto-generated.
# XXX:
# XXX: Source file: "cli/config/configuration.toml.in"
# XXX: Project:
# XXX:   Name: Kata Containers
# XXX:   Type: kata

[hypervisor.qemu]
path = "/usr/bin/qemu-lite-system-x86_64"
kernel = "/usr/share/kata-containers/vmlinuz.container"
image = "/usr/share/kata-containers/kata-containers.img"
#image = "/home/leilei.lin/go/src/github.com/kata-containers/osbuilder/image-builder/kata-containers.img"
machine_type = "pc"

# Optional space-separated list of options to pass to the guest kernel.
# For example, use `kernel_params = "vsyscall=emulate"` if you are having
# trouble running pre-2.15 glibc.
#
# WARNING: - any parameter specified here will take priority over the default
# parameter value of the same name used to start the virtual machine.
# Do not set values here unless you understand the impact of doing so as you
# may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a
# container and look for 'default-kernel-parameters' log entries.
kernel_params = "agent.log=debug"

# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = ""

# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
machine_accelerators=""

# Default number of vCPUs per SB/VM:
# unspecified or 0                --> will be set to 1
# < 0                             --> will be set to the actual number of physical cores
# > 0 <= number of physical cores --> will be set to the specified number
# > number of physical cores      --> will be set to the actual number of physical cores
default_vcpus = 1

# Default maximum number of vCPUs per SB/VM:
# unspecified or == 0             --> will be set to the actual number of physical cores or to the maximum number
#                                     of vCPUs supported by KVM if that number is exceeded
# > 0 <= number of physical cores --> will be set to the specified number
# > number of physical cores      --> will be set to the actual number of physical cores or to the maximum number
#                                     of vCPUs supported by KVM if that number is exceeded
# WARNING: Depending of the architecture, the maximum number of vCPUs supported by KVM is used when
# the actual number of physical cores is greater than it.
# WARNING: Be aware that this value impacts the virtual machine's memory footprint and CPU
# the hotplug functionality. For example, `default_maxvcpus = 240` specifies that until 240 vCPUs
# can be added to a SB/VM, but the memory footprint will be big. Another example, with
# `default_maxvcpus = 8` the memory footprint will be small, but 8 will be the maximum number of
# vCPUs supported by the SB/VM. In general, we recommend that you do not edit this variable,
# unless you know what are you doing.
default_maxvcpus = 0

# Bridges can be used to hot plug devices.
# Limitations:
# * Currently only pci bridges are supported
# * Until 30 devices per bridge can be hot plugged.
# * Until 5 PCI bridges can be cold plugged per VM.
#   This limitation could be a bug in qemu or in the kernel
# Default number of bridges per SB/VM:
# unspecified or 0   --> will be set to 1
# > 1 <= 5           --> will be set to the specified number
# > 5                --> will be set to 5
default_bridges = 1

# Default memory size in MiB for SB/VM.
# If unspecified then it will be set 2048 MiB.
#default_memory = 2048
#
# Default memory slots per SB/VM.
# If unspecified then it will be set 10.
# This is will determine the times that memory will be hotadded to sandbox/VM.
#memory_slots = 10

# Disable block device from being used for a container's rootfs.
# In case of a storage driver like devicemapper where a container's 
# root file system is backed by a block device, the block device is passed
# directly to the hypervisor for performance reasons. 
# This flag prevents the block device from being passed to the hypervisor, 
# 9pfs is used instead to pass the rootfs.
disable_block_device_use = false

# Block storage driver to be used for the hypervisor in case the container
# rootfs is backed by a block device. This is either virtio-scsi or 
# virtio-blk.
block_device_driver = "virtio-scsi"

# Enable iothreads (data-plane) to be used. This causes IO to be
# handled in a separate IO thread. This is currently only implemented
# for SCSI.
#
enable_iothreads = false

# Enable pre allocation of VM RAM, default false
# Enabling this will result in lower container density
# as all of the memory will be allocated and locked
# This is useful when you want to reserve all the memory
# upfront or in the cases where you want memory latencies
# to be very predictable
# Default false
#enable_mem_prealloc = true

# Enable huge pages for VM RAM, default false
# Enabling this will result in the VM memory
# being allocated using huge pages.
# This is useful when you want to use vhost-user network
# stacks within the container. This will automatically 
# result in memory pre allocation
#enable_hugepages = true

# Enable swap of vm memory. Default false.
# The behaviour is undefined if mem_prealloc is also set to true
#enable_swap = true

# This option changes the default hypervisor and kernel parameters
# to enable debug output where available. This extra output is added
# to the proxy logs, but only when proxy debug is also enabled.
# 
# Default false
enable_debug = true

# Disable the customizations done in the runtime when it detects
# that it is running on top a VMM. This will result in the runtime
# behaving as it would when running on bare metal.
# 
#disable_nesting_checks = true

# This is the msize used for 9p shares. It is the number of bytes 
# used for 9p packet payload.
#msize_9p = 8192

# If true and vsocks are supported, use vsocks to communicate directly
# with the agent and no proxy is started, otherwise use unix
# sockets and start a proxy to communicate with the agent.
# Default false
#use_vsock = true

# VFIO devices are hotplugged on a bridge by default. 
# Enable hotplugging on root bus. This may be required for devices with
# a large PCI bar, as this is a current limitation with hotplugging on 
# a bridge. This value is valid for "pc" machine type.
# Default false
#hotplug_vfio_on_root_bus = true

# If host doesn't support vhost_net, set to true. Thus we won't create vhost fds for nics.
# Default false
disable_vhost_net = true
#
# Default entropy source.
# The path to a host source of entropy (including a real hardware RNG)
# /dev/urandom and /dev/random are two main options.
# Be aware that /dev/random is a blocking source of entropy.  If the host
# runs out of entropy, the VMs boot time will increase leading to get startup
# timeouts.
# The source of entropy /dev/urandom is non-blocking and provides a
# generally acceptable source of entropy. It should work well for pretty much
# all practical purposes.
#entropy_source= "/dev/urandom"

[factory]
# VM templating support. Once enabled, new VMs are created from template
# using vm cloning. They will share the same initial kernel, initramfs and
# agent memory by mapping it readonly. It helps speeding up new container
# creation and saves a lot of memory if there are many kata containers running
# on the same host.
#
# When disabled, new VMs are created from scratch.
#
# Default false
enable_template = false

[proxy.kata]
path = "/usr/libexec/kata-containers/kata-proxy"

# If enabled, proxy messages will be sent to the system log
# (default: disabled)
enable_debug = true

[shim.kata]
path = "/usr/libexec/kata-containers/kata-shim"

# If enabled, shim messages will be sent to the system log
# (default: disabled)
enable_debug = true

[agent.kata]
# There is no field for this section. The goal is only to be able to
# specify which type of agent the user wants to use.

[netmon]
# If enabled, the network monitoring process gets started when the
# sandbox is created. This allows for the detection of some additional
# network being added to the existing network namespace, after the
# sandbox has been created.
# (default: disabled)
#enable_netmon = true

# Specify the path to the netmon binary.
path = "/usr/libexec/kata-containers/kata-netmon"

# If enabled, netmon messages will be sent to the system log
# (default: disabled)
#enable_debug = true

[runtime]
# If enabled, the runtime will log additional debug messages to the
# system log
# (default: disabled)
enable_debug = true
#
# Internetworking model
# Determines how the VM should be connected to the
# the container network interface
# Options:
#
#   - bridged
#     Uses a linux bridge to interconnect the container interface to
#     the VM. Works for most cases except macvlan and ipvlan.
#
#   - macvtap
#     Used when the Container network interface can be bridged using
#     macvtap.
internetworking_model="macvtap"

# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)
#enable_tracing = true

Output of "cat "/usr/share/defaults/kata-containers/configuration.toml"":

# Copyright (c) 2017-2018 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
#

# XXX: WARNING: this file is auto-generated.
# XXX:
# XXX: Source file: "cli/config/configuration.toml.in"
# XXX: Project:
# XXX:   Name: Kata Containers
# XXX:   Type: kata

[hypervisor.qemu]
path = "/usr/bin/qemu-lite-system-x86_64"
kernel = "/usr/share/kata-containers/vmlinuz.container"
image = "/usr/share/kata-containers/kata-containers.img"
machine_type = "pc"

# Optional space-separated list of options to pass to the guest kernel.
# For example, use `kernel_params = "vsyscall=emulate"` if you are having
# trouble running pre-2.15 glibc.
#
# WARNING: - any parameter specified here will take priority over the default
# parameter value of the same name used to start the virtual machine.
# Do not set values here unless you understand the impact of doing so as you
# may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a
# container and look for 'default-kernel-parameters' log entries.
kernel_params = ""

# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = ""

# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
machine_accelerators=""

# Default number of vCPUs per SB/VM:
# unspecified or 0                --> will be set to 1
# < 0                             --> will be set to the actual number of physical cores
# > 0 <= number of physical cores --> will be set to the specified number
# > number of physical cores      --> will be set to the actual number of physical cores
default_vcpus = 1

# Default maximum number of vCPUs per SB/VM:
# unspecified or == 0             --> will be set to the actual number of physical cores or to the maximum number
#                                     of vCPUs supported by KVM if that number is exceeded
# > 0 <= number of physical cores --> will be set to the specified number
# > number of physical cores      --> will be set to the actual number of physical cores or to the maximum number
#                                     of vCPUs supported by KVM if that number is exceeded
# WARNING: Depending of the architecture, the maximum number of vCPUs supported by KVM is used when
# the actual number of physical cores is greater than it.
# WARNING: Be aware that this value impacts the virtual machine's memory footprint and CPU
# the hotplug functionality. For example, `default_maxvcpus = 240` specifies that until 240 vCPUs
# can be added to a SB/VM, but the memory footprint will be big. Another example, with
# `default_maxvcpus = 8` the memory footprint will be small, but 8 will be the maximum number of
# vCPUs supported by the SB/VM. In general, we recommend that you do not edit this variable,
# unless you know what are you doing.
default_maxvcpus = 0

# Bridges can be used to hot plug devices.
# Limitations:
# * Currently only pci bridges are supported
# * Until 30 devices per bridge can be hot plugged.
# * Until 5 PCI bridges can be cold plugged per VM.
#   This limitation could be a bug in qemu or in the kernel
# Default number of bridges per SB/VM:
# unspecified or 0   --> will be set to 1
# > 1 <= 5           --> will be set to the specified number
# > 5                --> will be set to 5
default_bridges = 1

# Default memory size in MiB for SB/VM.
# If unspecified then it will be set 2048 MiB.
default_memory = 2048
#
# Default memory slots per SB/VM.
# If unspecified then it will be set 10.
# This is will determine the times that memory will be hotadded to sandbox/VM.
#memory_slots = 10

# Disable block device from being used for a container's rootfs.
# In case of a storage driver like devicemapper where a container's 
# root file system is backed by a block device, the block device is passed
# directly to the hypervisor for performance reasons. 
# This flag prevents the block device from being passed to the hypervisor, 
# 9pfs is used instead to pass the rootfs.
disable_block_device_use = false

# Block storage driver to be used for the hypervisor in case the container
# rootfs is backed by a block device. This is either virtio-scsi or 
# virtio-blk.
block_device_driver = "virtio-scsi"

# Enable iothreads (data-plane) to be used. This causes IO to be
# handled in a separate IO thread. This is currently only implemented
# for SCSI.
#
enable_iothreads = false

# Enable pre allocation of VM RAM, default false
# Enabling this will result in lower container density
# as all of the memory will be allocated and locked
# This is useful when you want to reserve all the memory
# upfront or in the cases where you want memory latencies
# to be very predictable
# Default false
#enable_mem_prealloc = true

# Enable huge pages for VM RAM, default false
# Enabling this will result in the VM memory
# being allocated using huge pages.
# This is useful when you want to use vhost-user network
# stacks within the container. This will automatically 
# result in memory pre allocation
#enable_hugepages = true

# Enable swap of vm memory. Default false.
# The behaviour is undefined if mem_prealloc is also set to true
#enable_swap = true

# This option changes the default hypervisor and kernel parameters
# to enable debug output where available. This extra output is added
# to the proxy logs, but only when proxy debug is also enabled.
# 
# Default false
#enable_debug = true

# Disable the customizations done in the runtime when it detects
# that it is running on top a VMM. This will result in the runtime
# behaving as it would when running on bare metal.
# 
#disable_nesting_checks = true

# This is the msize used for 9p shares. It is the number of bytes 
# used for 9p packet payload.
#msize_9p = 8192

# If true and vsocks are supported, use vsocks to communicate directly
# with the agent and no proxy is started, otherwise use unix
# sockets and start a proxy to communicate with the agent.
# Default false
#use_vsock = true

# VFIO devices are hotplugged on a bridge by default. 
# Enable hotplugging on root bus. This may be required for devices with
# a large PCI bar, as this is a current limitation with hotplugging on 
# a bridge. This value is valid for "pc" machine type.
# Default false
#hotplug_vfio_on_root_bus = true

# If host doesn't support vhost_net, set to true. Thus we won't create vhost fds for nics.
# Default false
#disable_vhost_net = true
#
# Default entropy source.
# The path to a host source of entropy (including a real hardware RNG)
# /dev/urandom and /dev/random are two main options.
# Be aware that /dev/random is a blocking source of entropy.  If the host
# runs out of entropy, the VMs boot time will increase leading to get startup
# timeouts.
# The source of entropy /dev/urandom is non-blocking and provides a
# generally acceptable source of entropy. It should work well for pretty much
# all practical purposes.
#entropy_source= "/dev/urandom"

# Path to OCI hook binaries in the *guest rootfs*.
# This does not affect host-side hooks which must instead be added to
# the OCI spec passed to the runtime.
#
# You can create a rootfs with hooks by customizing the osbuilder scripts:
# https://github.com/kata-containers/osbuilder
#
# Hooks must be stored in a subdirectory of guest_hook_path according to their
# hook type, i.e. "guest_hook_path/{prestart,postart,poststop}".
# The agent will scan these directories for executable files and add them, in
# lexicographical order, to the lifecycle of the guest container.
# Hooks are executed in the runtime namespace of the guest. See the official documentation:
# https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#posix-platform-hooks
# Warnings will be logged if any error is encountered will scanning for hooks,
# but it will not abort container execution.
#guest_hook_path = "/usr/share/oci/hooks"

[factory]
# VM templating support. Once enabled, new VMs are created from template
# using vm cloning. They will share the same initial kernel, initramfs and
# agent memory by mapping it readonly. It helps speeding up new container
# creation and saves a lot of memory if there are many kata containers running
# on the same host.
#
# When disabled, new VMs are created from scratch.
#
# Default false
#enable_template = true

[proxy.kata]
path = "/usr/libexec/kata-containers/kata-proxy"

# If enabled, proxy messages will be sent to the system log
# (default: disabled)
#enable_debug = true

[shim.kata]
path = "/usr/libexec/kata-containers/kata-shim"

# If enabled, shim messages will be sent to the system log
# (default: disabled)
#enable_debug = true

[agent.kata]
# There is no field for this section. The goal is only to be able to
# specify which type of agent the user wants to use.

[netmon]
# If enabled, the network monitoring process gets started when the
# sandbox is created. This allows for the detection of some additional
# network being added to the existing network namespace, after the
# sandbox has been created.
# (default: disabled)
#enable_netmon = true

# Specify the path to the netmon binary.
path = "/usr/libexec/kata-containers/kata-netmon"

# If enabled, netmon messages will be sent to the system log
# (default: disabled)
#enable_debug = true

[runtime]
# If enabled, the runtime will log additional debug messages to the
# system log
# (default: disabled)
#enable_debug = true
#
# Internetworking model
# Determines how the VM should be connected to the
# the container network interface
# Options:
#
#   - bridged
#     Uses a linux bridge to interconnect the container interface to
#     the VM. Works for most cases except macvlan and ipvlan.
#
#   - macvtap
#     Used when the Container network interface can be bridged using
#     macvtap.
#
#   - none
#     Used when customize network. Only creates a tap device. No veth pair.
#
#   - tcfilter
#     Uses tc filter rules to redirect traffic from the network interface
#     provided by plugin to a tap interface connected to the VM.
#
internetworking_model="macvtap"

# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)
#enable_tracing = true

# If enabled, the runtime will not create a network namespace for shim and hypervisor processes.
# This option may have some potential impacts to your host. It should only be used when you know what you're doing.
# `disable_new_netns` conflicts with `enable_netmon`
# `disable_new_netns` conflicts with `internetworking_model=bridged` and `internetworking_model=macvtap`. It works only
# with `internetworking_model=none`. The tap device will be in the host network namespace and can connect to a bridge
# (like OVS) directly.
# If you are using docker, `disable_new_netns` only works with `docker run --net=none`
# (default: false)
#disable_new_netns = true

---

KSM throttler

version

Output of "/usr/libexec/kata-ksm-throttler/kata-ksm-throttler --version":

kata-ksm-throttler version 1.1.0-1fecaff

Output of "/usr/lib/systemd/system/kata-ksm-throttler.service --version":

/usr/bin/kata-collect-data.sh: line 168: /usr/lib/systemd/system/kata-ksm-throttler.service: Permission denied

systemd service

Image details

---
osbuilder:
  url: "https://github.com/kata-containers/osbuilder"
  version: "unknown"
rootfs-creation-time: "2018-10-22T21:13:25.475975441+0000Z"
description: "osbuilder rootfs"
file-format-version: "0.0.2"
architecture: "x86_64"
base-distro:
  name: "Clear"
  version: "25740"
  packages:
    default:
      - "iptables-bin"
      - "libudev0-shim"
      - "systemd"
    extra:

agent:
  url: "https://github.com/kata-containers/agent"
  name: "kata-agent"
  version: "1.3.1-c7fdd324cda8e2ef01203a86d97b03a392e6eb39"
  agent-is-init-daemon: "no"

---

Initrd details

No initrd

---

Logfiles

Runtime logs

Recent runtime problems found in system journal:

time="2018-11-28T17:17:18.812497534+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.465065] systemd[75]: Failed to add rule for system call fadvise64_64() / -10007, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.812681309+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.465256] systemd[75]: Failed to add rule for system call setregid32() / -10048, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.812841925+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.465437] systemd[75]: Failed to add rule for system call statfs64() / -10063, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.813001738+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.465597] systemd[75]: Failed to add rule for system call ipc() / -10024, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.813192031+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.465761] systemd[75]: Failed to add rule for system call fstatat64() / -10011, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.813352396+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.465949] systemd[75]: Failed to add rule for system call nice() / -10033, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.814455344+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.467047] systemd[75]: Failed to add rule for system call lchown32() / -10025, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.814651985+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.467226] systemd[75]: Failed to add rule for system call pciconfig_iobase() / -10086, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.814825103+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.467404] systemd[75]: Failed to add rule for system call sigreturn() / -10058, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.81499213+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.467585] systemd[75]: Failed to add rule for system call readdir() / -10041, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.815169541+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.467757] systemd[75]: Failed to add rule for system call set_tls() / -10183, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.815332487+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.467930] systemd[75]: Failed to add rule for system call setuid32() / -10052, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.817505261+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.470078] systemd[75]: Failed to add rule for system call oldolduname() / -10036, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.817694931+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.470271] systemd[75]: Failed to add rule for system call getresgid32() / -10019, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.817862388+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.470451] systemd[75]: Failed to add rule for system call oldstat() / -10037, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.818032354+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.470612] systemd[75]: Failed to add rule for system call sigsuspend() / -10059, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.818220308+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.470796] systemd[75]: Failed to add rule for system call ftruncate64() / -10014, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.818381618+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.470982] systemd[75]: Failed to add rule for system call fcntl64() / -10009, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.821509928+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.474077] systemd[75]: Failed to add rule for system call geteuid32() / -10016, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.821668468+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.474265] systemd[75]: Failed to add rule for system call signal() / -10055, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.821840718+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.474434] systemd[75]: Failed to add rule for system call recv() / -110, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.82201857+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.474606] systemd[75]: Failed to add rule for system call mmap2() / -10029, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.822203406+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.474777] systemd[75]: Failed to add rule for system call sigprocmask() / -10057, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.82238216+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.474962] systemd[75]: Failed to add rule for system call setfsuid32() / -10045, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.825244007+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.477825] systemd[75]: Failed to add rule for system call oldlstat() / -10035, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.825447551+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.478038] systemd[75]: Failed to add rule for system call getgid32() / -10017, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.825615682+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.478209] systemd[75]: Failed to add rule for system call waitpid() / -10073, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.825797559+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.478371] systemd[75]: Failed to add rule for system call pciconfig_read() / -10087, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.825962172+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.478552] systemd[75]: Failed to add rule for system call chown32() / -10004, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.826138758+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.478713] systemd[75]: Failed to add rule for system call sigpending() / -10056, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.826324504+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.478902] systemd[75]: Failed to add rule for system call s390_pci_mmio_write() / -10198, ignoring: Numerical argument out of domain"
time="2018-11-28T17:17:18.826785972+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[    0.479381] systemd-journald[69]: Failed to open configuration file '/etc/systemd/journald.conf': No such file or directory"
time="2018-11-28T17:17:19.009554404+08:00" level=debug msg="reading guest console" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 sandbox=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9 vmconsole="[\x1b[0;1;31mFAILED\x1b[0m] Failed to start Login Service."
time="2018-11-28T17:17:37.923642336+08:00" level=info msg="clean up proxy" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 error="Failed to check if grpc server is working: rpc error: code = Unavailable desc = transport is closing" name=kata-runtime pid=7706 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9
time="2018-11-28T17:17:37.923797557+08:00" level=error msg="Failed to read agent logs" arch=amd64 command=create console-protocol=unix console-socket=/run/vc/vm/9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9/console.sock container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 error="read unix @->/run/vc/vm/9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9/console.sock: use of closed network connection" name=kata-runtime pid=7706 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9
time="2018-11-28T17:17:37.923887565+08:00" level=info msg="clean up vm" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 error="Failed to check if grpc server is working: rpc error: code = Unavailable desc = transport is closing" name=kata-runtime pid=7706 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9
time="2018-11-28T17:17:37.925331846+08:00" level=error msg="failed to create new vm" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 error="Failed to check if grpc server is working: rpc error: code = Unavailable desc = transport is closing" name=kata-runtime pid=7706 source=virtcontainers vm=9a5f7ef6-1bec-42a2-bc90-f5acf8c22ac9
time="2018-11-28T17:17:38.035933448+08:00" level=debug msg="Could not retrieve anything from storage" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 source=virtcontainers subsystem=kata_agent
time="2018-11-28T17:17:38.040160378+08:00" level=warning msg="fetch sandbox device failed" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 error="open /run/vc/sbs/d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64/devices.json: no such file or directory" name=kata-runtime pid=7706 sandbox=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 sandboxid=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 source=virtcontainers subsystem=sandbox
time="2018-11-28T17:17:38.056283353+08:00" level=info msg="fallback to direct factory vm" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 error="hypervisor config does not match, base: {HypervisorType:qemu HypervisorConfig:{NumVCPUs:0 DefaultMaxVCPUs:32 MemorySize:0 DefaultBridges:1 Msize9p:8192 MemSlots:10 KernelParams:[{Key:init Value:/usr/lib/systemd/systemd} {Key:systemd.unit Value:kata-containers.target} {Key:systemd.mask Value:systemd-networkd.service} {Key:systemd.mask Value:systemd-networkd.socket} {Key:agent.log Value:debug}] HypervisorParams:[] KernelPath:/usr/share/kata-containers/vmlinuz-4.14.67-19.7.container ImagePath:/usr/share/kata-containers/kata-containers-image_clearlinux_1.4.0_agent_0aae82b.img InitrdPath: FirmwarePath: MachineAccelerators: HypervisorPath:/usr/bin/qemu-lite-system-x86_64 BlockDeviceDriver:virtio-scsi HypervisorMachineType:pc MemoryPath: DevicesStatePath: EntropySource:/dev/urandom customAssets:map[] DisableBlockDeviceUse:false EnableIOThreads:false Debug:true MemPrealloc:false HugePages:false Realtime:false Mlock:true DisableNestingChecks:false UseVSock:false HotplugVFIOOnRootBus:false BootToBeTemplate:false BootFromTemplate:false DisableVhostNet:true GuestHookPath:} AgentType:kata AgentConfig:{LongLiveConn:false UseVSock:false} ProxyType:noopProxy ProxyConfig:{Path: Debug:false}}. new: {HypervisorType:qemu HypervisorConfig:{NumVCPUs:0 DefaultMaxVCPUs:32 MemorySize:0 DefaultBridges:1 Msize9p:8192 MemSlots:10 KernelParams:[{Key:agent.log Value:debug}] HypervisorParams:[] KernelPath:/usr/share/kata-containers/vmlinuz-4.14.67-19.7.container ImagePath:/usr/share/kata-containers/kata-containers-image_clearlinux_1.4.0_agent_0aae82b.img InitrdPath: FirmwarePath: MachineAccelerators: HypervisorPath:/usr/bin/qemu-lite-system-x86_64 BlockDeviceDriver:virtio-scsi HypervisorMachineType:pc MemoryPath: DevicesStatePath: EntropySource:/dev/urandom customAssets:map[] DisableBlockDeviceUse:false EnableIOThreads:false Debug:true MemPrealloc:false HugePages:false Realtime:false Mlock:true DisableNestingChecks:false UseVSock:false HotplugVFIOOnRootBus:false BootToBeTemplate:false BootFromTemplate:false DisableVhostNet:true GuestHookPath:} AgentType:kata AgentConfig:{LongLiveConn:false UseVSock:false} ProxyType:noopProxy ProxyConfig:{Path: Debug:false}}" name=kata-runtime pid=7706 source=virtcontainers subsystem=factory
time="2018-11-28T17:17:38.056956556+08:00" level=debug arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 debug systemd.show_status=true systemd.log_level=debug" name=kata-runtime pid=7706 source=virtcontainers subsystem=qemu
time="2018-11-28T17:17:38.102135051+08:00" level=error msg="trace called before context set" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 source=virtcontainers subsystem=kata_agent type=bug
time="2018-11-28T17:17:59.186750808+08:00" level=error msg="Failed to check if grpc server is working: rpc error: code = Unavailable desc = transport is closing" arch=amd64 command=create container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=7706 source=runtime
time="2018-11-28T17:17:59.210483904+08:00" level=error msg="Container ID (d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64) does not exist" arch=amd64 command=delete container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=8691 source=runtime
time="2018-11-28T17:17:59.258397699+08:00" level=error msg="Container ID (d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64) does not exist" arch=amd64 command=delete container=d9b7c02fbe1d6fc7ceca900a1d579e5135f9b6c4d45a890759fd26ec9ac2df64 name=kata-runtime pid=8701 source=runtime
time="2018-11-28T17:18:02.557134141+08:00" level=debug msg="Could not retrieve anything from storage" arch=amd64 command=create container=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 name=kata-runtime pid=8762 source=virtcontainers subsystem=kata_agent
time="2018-11-28T17:18:02.561430251+08:00" level=warning msg="fetch sandbox device failed" arch=amd64 command=create container=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 error="open /run/vc/sbs/fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8/devices.json: no such file or directory" name=kata-runtime pid=8762 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 sandboxid=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=virtcontainers subsystem=sandbox
time="2018-11-28T17:18:02.629123094+08:00" level=debug arch=amd64 command=create container=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 debug systemd.show_status=true systemd.log_level=debug" name=kata-runtime pid=8762 source=virtcontainers subsystem=qemu
time="2018-11-28T18:51:11.794746181+08:00" level=error msg="Container not ready, running or paused, impossible to signal the container" arch=amd64 command=kill container=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 name=kata-runtime pid=84199 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=runtime
time="2018-11-28T18:51:11.887288075+08:00" level=error msg="Container ID (fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8) does not exist" arch=amd64 command=delete container=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 name=kata-runtime pid=84238 source=runtime

Proxy logs

Recent proxy problems found in system journal:

time="2018-11-28T17:18:03.464020024+08:00" level=info msg="[    0.423040] systemd[78]: Failed to add rule for system call sigreturn() / -10058, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.464190996+08:00" level=info msg="[    0.423223] systemd[78]: Failed to add rule for system call set_tls() / -10183, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.464375598+08:00" level=info msg="[    0.423397] systemd[78]: Failed to add rule for system call s390_pci_mmio_read() / -10197, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.464556048+08:00" level=info msg="[    0.423574] systemd[78]: Failed to add rule for system call setresgid32() / -10049, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.464717026+08:00" level=info msg="[    0.423757] systemd[78]: Failed to add rule for system call send() / -109, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.464892595+08:00" level=info msg="[    0.423917] systemd[78]: Failed to add rule for system call socketcall() / -10060, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.465078674+08:00" level=info msg="[    0.424115] systemd[69]: Failed to add rule for system call oldfstat() / -10034, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.465250022+08:00" level=info msg="[    0.424283] systemd[69]: Failed to add rule for system call mmap2() / -10029, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.465420773+08:00" level=info msg="[    0.424443] systemd[69]: Failed to add rule for system call fstatat64() / -10011, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.467011694+08:00" level=info msg="[    0.426046] systemd[78]: Failed to add rule for system call oldstat() / -10037, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.467179352+08:00" level=info msg="[    0.426213] systemd[78]: Failed to add rule for system call oldfstat() / -10034, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.467339449+08:00" level=info msg="[    0.426381] systemd[78]: Failed to add rule for system call lchown32() / -10025, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.467513714+08:00" level=info msg="[    0.426539] systemd[78]: Failed to add rule for system call fstatfs64() / -10012, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.467677124+08:00" level=info msg="[    0.426717] systemd[78]: Failed to add rule for system call olduname() / -10038, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.467844221+08:00" level=info msg="[    0.426874] systemd[78]: Failed to add rule for system call getgid32() / -10017, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.470023467+08:00" level=info msg="[    0.429057] systemd[78]: Failed to add rule for system call chown32() / -10004, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.470209483+08:00" level=info msg="[    0.429227] systemd[78]: Failed to add rule for system call setfsuid32() / -10045, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.470371799+08:00" level=info msg="[    0.429411] systemd[78]: Failed to add rule for system call recv() / -110, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.470545853+08:00" level=info msg="[    0.429567] systemd[78]: Failed to add rule for system call pciconfig_write() / -10088, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.470718757+08:00" level=info msg="[    0.429756] systemd[78]: Failed to add rule for system call mmap2() / -10029, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.470890397+08:00" level=info msg="[    0.429913] systemd[78]: Failed to add rule for system call oldolduname() / -10036, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.474030589+08:00" level=info msg="[    0.433047] systemd[78]: Failed to add rule for system call getgroups32() / -10018, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.474220996+08:00" level=info msg="[    0.433231] systemd[78]: Failed to add rule for system call sendfile64() / -10043, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.474395505+08:00" level=info msg="[    0.433419] systemd[78]: Failed to add rule for system call pciconfig_iobase() / -10086, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.474560376+08:00" level=info msg="[    0.433597] systemd[78]: Failed to add rule for system call setuid32() / -10052, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.474742449+08:00" level=info msg="[    0.433764] systemd[78]: Failed to add rule for system call sigpending() / -10056, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.474902901+08:00" level=info msg="[    0.433944] systemd[78]: Failed to add rule for system call fchown32() / -10008, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.477012326+08:00" level=info msg="[    0.436046] systemd[78]: Failed to add rule for system call _llseek() / -10026, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.477196711+08:00" level=info msg="[    0.436212] systemd[78]: Failed to add rule for system call getresuid32() / -10020, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.477380009+08:00" level=info msg="[    0.436401] systemd[78]: Failed to add rule for system call pciconfig_read() / -10087, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.477556226+08:00" level=info msg="[    0.436579] systemd[78]: Failed to add rule for system call truncate64() / -10066, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.477752234+08:00" level=info msg="[    0.436774] systemd[78]: Failed to add rule for system call fstatat64() / -10011, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.477913142+08:00" level=info msg="[    0.436952] systemd[78]: Failed to add rule for system call nice() / -10033, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.479269776+08:00" level=info msg="[    0.438301] systemd-journald[69]: Failed to open configuration file '/etc/systemd/journald.conf': No such file or directory\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.480035068+08:00" level=info msg="[    0.439060] systemd[78]: Failed to add rule for system call getuid32() / -10021, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.480192386+08:00" level=info msg="[    0.439225] systemd[78]: Failed to add rule for system call statfs64() / -10063, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.480370469+08:00" level=info msg="[    0.439394] systemd[78]: Failed to add rule for system call ugetrlimit() / -10068, ignoring: Numerical argument out of domain\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.635421852+08:00" level=info msg="time=\"2018-11-28T09:18:03.549059687Z\" level=debug msg=\"request end\" duration=6.402544ms name=kata-agent pid=97 request=/grpc.AgentService/CreateSandbox resp=\"&Empty{}\" sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.744811187+08:00" level=info msg="time=\"2018-11-28T09:18:03.658433158Z\" level=debug msg=\"request end\" duration=95.601835ms name=kata-agent pid=97 request=/grpc.AgentService/CreateContainer resp=\"&Empty{}\" sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.745015276+08:00" level=info msg="time=\"2018-11-28T09:18:03.658618716Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 signal=\"child exited\" source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.824976948+08:00" level=info msg="time=\"2018-11-28T09:18:03.73860509Z\" level=debug msg=\"request end\" duration=40.00592ms name=kata-agent pid=97 request=/grpc.AgentService/StartContainer resp=\"&Empty{}\" sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T17:18:03.853831491+08:00" level=info msg="time=\"2018-11-28T09:18:03.767456959Z\" level=debug msg=\"request end\" duration=\"9.508µs\" name=kata-agent pid=97 request=/grpc.AgentService/TtyWinResize resp=\"&Empty{}\" sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T18:51:11.760895159+08:00" level=info msg="time=\"2018-11-28T10:51:11.576788737Z\" level=debug msg=\"request end\" duration=182.930202ms error=\"read /dev/ptmx: input/output error\" name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp= sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T18:51:11.76158675+08:00" level=info msg="time=\"2018-11-28T10:51:11.577470002Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 signal=\"child exited\" source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T18:51:11.769172953+08:00" level=debug msg="Copy stream error" error="read unix /run/vc/sbs/fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8/proxy.sock->@: read: connection reset by peer" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=proxy
time="2018-11-28T18:51:11.785170796+08:00" level=info msg="time=\"2018-11-28T10:51:11.601090869Z\" level=debug msg=\"request end\" duration=\"145.334µs\" name=kata-agent pid=97 request=/grpc.AgentService/SignalProcess resp=\"&Empty{}\" sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T18:51:11.786642504+08:00" level=info msg="time=\"2018-11-28T10:51:11.602549755Z\" level=debug msg=\"request end\" duration=\"4.069µs\" error=\"rpc error: code = NotFound desc = Process fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 not found (container fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8)\" name=kata-agent pid=97 request=/grpc.AgentService/WaitProcess resp= sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T18:51:11.788355293+08:00" level=info msg="time=\"2018-11-28T10:51:11.604276279Z\" level=debug msg=\"request end\" duration=\"410.744µs\" name=kata-agent pid=97 request=/grpc.AgentService/RemoveContainer resp=\"&Empty{}\" sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T18:51:11.83395185+08:00" level=info msg="time=\"2018-11-28T10:51:11.649868698Z\" level=debug msg=\"request end\" duration=3.904935ms name=kata-agent pid=97 request=/grpc.AgentService/DestroySandbox resp=\"&Empty{}\" sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent\n" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=agent
time="2018-11-28T18:51:11.83434358+08:00" level=fatal msg="channel error" error="accept unix /run/vc/sbs/fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8/proxy.sock: use of closed network connection" name=kata-proxy pid=8817 sandbox=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 source=proxy

Shim logs

Recent shim problems found in system journal:

time="2018-11-27T18:03:12.069960617+08:00" level=info msg="copy stdout failed" container=1234 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=1234 name=kata-shim pid=1 source=shim
time="2018-11-27T18:03:17.121787295+08:00" level=info msg="copy stdout failed" container=1234 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=1234 name=kata-shim pid=1 source=shim
time="2018-11-27T18:05:09.389223055+08:00" level=info msg="copy stdout failed" container=1234 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=1234 name=kata-shim pid=1 source=shim
time="2018-11-27T18:05:41.880634512+08:00" level=info msg="copy stdout failed" container=740ac1ac9007826d23a2e975f1518d267696beb2d0b8a4536af58c536a6f28d2 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=740ac1ac9007826d23a2e975f1518d267696beb2d0b8a4536af58c536a6f28d2 name=kata-shim pid=1 source=shim
time="2018-11-27T18:05:53.467964331+08:00" level=info msg="copy stdout failed" container=4b007660b98bb2650bf1c6e76eba1e1928b518ed0f7a395407bec88b2cd71b9a error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=4b007660b98bb2650bf1c6e76eba1e1928b518ed0f7a395407bec88b2cd71b9a name=kata-shim pid=1 source=shim
time="2018-11-27T18:06:07.220167881+08:00" level=info msg="copy stdout failed" container=2922f3cd4e6a39c74e69dbc1974b57132acd460687e5eebe52392e230686e122 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=2922f3cd4e6a39c74e69dbc1974b57132acd460687e5eebe52392e230686e122 name=kata-shim pid=1 source=shim
time="2018-11-27T18:06:17.214405615+08:00" level=info msg="copy stdout failed" container=fd2bc50831357bc28e50d90b5df54afcd223fbc458182476ebcf64472b70e25d error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=fd2bc50831357bc28e50d90b5df54afcd223fbc458182476ebcf64472b70e25d name=kata-shim pid=1 source=shim
time="2018-11-27T18:06:45.51081395+08:00" level=error msg="failed to set raw terminal" container=1 error="inappropriate ioctl for device" exec-id=1 name=kata-shim pid=1 source=shim
time="2018-11-27T18:08:06.332466238+08:00" level=info msg="copy stdout failed" container=8eb2d00db540b36045c68dd0ffaa7fe57ab65141a0b3d52bd943110d4f1b1bf1 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=8eb2d00db540b36045c68dd0ffaa7fe57ab65141a0b3d52bd943110d4f1b1bf1 name=kata-shim pid=1 source=shim
time="2018-11-27T18:08:16.192482717+08:00" level=error msg="failed to set raw terminal" container=2 error="inappropriate ioctl for device" exec-id=2 name=kata-shim pid=1 source=shim
time="2018-11-27T18:10:39.80992549+08:00" level=error msg="failed to set raw terminal" container=3 error="inappropriate ioctl for device" exec-id=3 name=kata-shim pid=1 source=shim
time="2018-11-27T18:25:00.990811026+08:00" level=error msg="failed to set raw terminal" container=4 error="inappropriate ioctl for device" exec-id=4 name=kata-shim pid=1 source=shim
time="2018-11-27T18:43:13.203727849+08:00" level=error msg="failed to set raw terminal" container=5 error="inappropriate ioctl for device" exec-id=5 name=kata-shim pid=1 source=shim
time="2018-11-27T19:56:04.640046272+08:00" level=error msg="failed to set raw terminal" container=5 error="inappropriate ioctl for device" exec-id=5 name=kata-shim pid=1 source=shim
time="2018-11-27T20:05:40.307462718+08:00" level=info msg="copy stdout failed" container=497ceacce3aa5fd96914ffa1ed41be656aa5704d693be1157798a5a78b1e6cd5 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=497ceacce3aa5fd96914ffa1ed41be656aa5704d693be1157798a5a78b1e6cd5 name=kata-shim pid=1 source=shim
time="2018-11-27T20:05:48.39159363+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:26:38.320947935+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:44:08.321825956+08:00" level=info msg="copy stdout failed" container=9bd7313c4a2bbbc00ebb7b334480ac18ab55d4633c4109f72da1f46465563c30 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=9bd7313c4a2bbbc00ebb7b334480ac18ab55d4633c4109f72da1f46465563c30 name=kata-shim pid=1 source=shim
time="2018-11-27T20:46:19.007470821+08:00" level=info msg="copy stdout failed" container=6 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:46:29.137055575+08:00" level=info msg="copy stdout failed" container=6 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:46:57.834277846+08:00" level=info msg="copy stdout failed" container=6 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:47:01.818227348+08:00" level=info msg="copy stdout failed" container=6 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:49:44.23464492+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:50:09.981243279+08:00" level=info msg="copy stdout failed" container=6 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:50:16.343012522+08:00" level=info msg="copy stdout failed" container=6 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:50:35.05571855+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:50:51.522628983+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:51:19.130153733+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T20:54:44.480841554+08:00" level=warning msg="copy stdin failed" container=7 error="read /dev/stdin: input/output error" exec-id=7 name=kata-shim pid=1 source=shim
time="2018-11-27T20:54:47.320587787+08:00" level=info msg="copy stdout failed" container=7 error="rpc error: code = Unknown desc = EOF" exec-id=7 name=kata-shim pid=1 source=shim
time="2018-11-27T20:54:47.320705578+08:00" level=info msg="copy stderr failed" container=7 error="rpc error: code = Unknown desc = EOF" exec-id=7 name=kata-shim pid=1 source=shim
time="2018-11-27T21:04:37.933220836+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T21:05:07.69969266+08:00" level=info msg="copy stdout failed" container=6 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T21:07:46.70493881+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T21:25:23.091257859+08:00" level=error msg="failed to set raw terminal" container=6 error="inappropriate ioctl for device" exec-id=6 name=kata-shim pid=1 source=shim
time="2018-11-27T21:47:36.272694453+08:00" level=info msg="copy stdout failed" container=25223d8d5498924bff08f77e8588e6eaecc2dac5653752783c570a781485174e error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=25223d8d5498924bff08f77e8588e6eaecc2dac5653752783c570a781485174e name=kata-shim pid=1 source=shim
time="2018-11-27T21:57:53.457814807+08:00" level=info msg="copy stdout failed" container=466c61958cd218164e473f4a650a28a1ea118813227cc789c944de5f24481029 error="rpc error: code = Unknown desc = EOF" exec-id=466c61958cd218164e473f4a650a28a1ea118813227cc789c944de5f24481029 name=kata-shim pid=1 source=shim
time="2018-11-27T21:57:53.457930849+08:00" level=info msg="copy stderr failed" container=466c61958cd218164e473f4a650a28a1ea118813227cc789c944de5f24481029 error="rpc error: code = Unknown desc = EOF" exec-id=466c61958cd218164e473f4a650a28a1ea118813227cc789c944de5f24481029 name=kata-shim pid=1 source=shim
time="2018-11-27T21:57:56.812206306+08:00" level=info msg="copy stdout failed" container=759c688d7dc9cc22851eab8ed14d65a6a85125badc97816b128a819c009c42fb error="rpc error: code = Unknown desc = EOF" exec-id=759c688d7dc9cc22851eab8ed14d65a6a85125badc97816b128a819c009c42fb name=kata-shim pid=1 source=shim
time="2018-11-27T21:57:56.812320474+08:00" level=info msg="copy stderr failed" container=759c688d7dc9cc22851eab8ed14d65a6a85125badc97816b128a819c009c42fb error="rpc error: code = Unknown desc = EOF" exec-id=759c688d7dc9cc22851eab8ed14d65a6a85125badc97816b128a819c009c42fb name=kata-shim pid=1 source=shim
time="2018-11-27T21:58:03.994147281+08:00" level=info msg="copy stdout failed" container=e0700a48c2ff22c087a434ba6aa5a51cac9ecc0da6beb28dcd89aba2663e1589 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=e0700a48c2ff22c087a434ba6aa5a51cac9ecc0da6beb28dcd89aba2663e1589 name=kata-shim pid=1 source=shim
time="2018-11-27T22:15:18.987520516+08:00" level=info msg="copy stdout failed" container=923147dec08da795748f2716eed62122ca0937e9f5e4da8fa9c4a754f8cd640a error="rpc error: code = Unknown desc = EOF" exec-id=923147dec08da795748f2716eed62122ca0937e9f5e4da8fa9c4a754f8cd640a name=kata-shim pid=1 source=shim
time="2018-11-27T22:15:18.987669408+08:00" level=info msg="copy stderr failed" container=923147dec08da795748f2716eed62122ca0937e9f5e4da8fa9c4a754f8cd640a error="rpc error: code = Unknown desc = EOF" exec-id=923147dec08da795748f2716eed62122ca0937e9f5e4da8fa9c4a754f8cd640a name=kata-shim pid=1 source=shim
time="2018-11-27T22:16:49.195604108+08:00" level=info msg="copy stdout failed" container=d6fc7150cf809f70a350201b2ce620bd239a2ceef81be037cd2412d24ab98e67 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=d6fc7150cf809f70a350201b2ce620bd239a2ceef81be037cd2412d24ab98e67 name=kata-shim pid=1 source=shim
time="2018-11-28T13:41:53.518521705+08:00" level=info msg="copy stdout failed" container=d24e2c53e937e85f616378877e87fd0cbd9bb6c5828b7bab6a908e0dc083b76a error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=d24e2c53e937e85f616378877e87fd0cbd9bb6c5828b7bab6a908e0dc083b76a name=kata-shim pid=1 source=shim
time="2018-11-28T16:23:48.088117384+08:00" level=info msg="copy stdout failed" container=7b245f27d4c60626f63a0bb90fb74680f810a958aebe0009f62d51c4f0b50923 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=7b245f27d4c60626f63a0bb90fb74680f810a958aebe0009f62d51c4f0b50923 name=kata-shim pid=1 source=shim
time="2018-11-28T16:33:41.624212285+08:00" level=info msg="copy stdout failed" container=1b8fcdc05d6641b81a36daae322b2239adaebc4921a22d68949a44d2002cd728 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=1b8fcdc05d6641b81a36daae322b2239adaebc4921a22d68949a44d2002cd728 name=kata-shim pid=1 source=shim
time="2018-11-28T16:58:08.314357305+08:00" level=info msg="copy stdout failed" container=e4601f14774d4300db1407391d979651074e5000281be6acd70fc30cd2fa7dba error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=e4601f14774d4300db1407391d979651074e5000281be6acd70fc30cd2fa7dba name=kata-shim pid=1 source=shim
time="2018-11-28T17:00:52.202993873+08:00" level=info msg="copy stdout failed" container=d14df36d891c785baa551c0aac2ad0e31efdce1399f2b18144f917a8250b6f27 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=d14df36d891c785baa551c0aac2ad0e31efdce1399f2b18144f917a8250b6f27 name=kata-shim pid=1 source=shim
time="2018-11-28T18:51:11.761155155+08:00" level=info msg="copy stdout failed" container=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=fcc9fd4261476f73fe6f45fa9c826e545bea8a5d1b50ce8d647c4f304cb997b8 name=kata-shim pid=1 source=shim

Throttler logs

No recent throttler problems found in system journal.

---

Container manager details

Have docker

Docker

Output of "docker version":

Client:
 Version:      17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:41:23 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.09.0-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:42:49 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of "docker info":

Containers: 236
 Running: 0
 Paused: 0
 Stopped: 236
Images: 27
Server Version: 17.09.0-ce
Storage Driver: overlay
 Backing Filesystem: extfs
 Supports d_type: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: cc-runtime.sh kata-runtime runc runsc-kvm runsc-ptrace cc-runtime
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 3.10.0-327.x86_64
Operating System: Linux Server 7.2 (Paladin)
OSType: linux
Architecture: x86_64
CPUs: 32
Total Memory: 125.7GiB
Name: e18g06460.et15sqa
ID: 4KV7:MCEK:3TU5:HN3X:EJIR:CL7H:DXOY:LYMS:7NYU:5MKM:TWRV:KLYL
Docker Root Dir: /mnt/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Registry Mirrors:
 https://registry.docker-cn.com/
Live Restore Enabled: false

Output of "systemctl show docker":

Type=notify
Restart=on-failure
NotifyAccess=main
RestartUSec=100ms
TimeoutStartUSec=0
TimeoutStopUSec=1min 30s
WatchdogUSec=0
WatchdogTimestamp=Tue 2018-11-27 17:29:06 CST
WatchdogTimestampMonotonic=10194333691359
StartLimitInterval=60000000
StartLimitBurst=3
StartLimitAction=none
FailureAction=none
PermissionsStartOnly=no
RootDirectoryStartOnly=no
RemainAfterExit=no
GuessMainPID=yes
MainPID=73827
ControlPID=0
FileDescriptorStoreMax=0
StatusErrno=0
Result=success
ExecMainStartTimestamp=Tue 2018-11-27 17:29:04 CST
ExecMainStartTimestampMonotonic=10194331578097
ExecMainExitTimestampMonotonic=0
ExecMainPID=73827
ExecMainCode=0
ExecMainStatus=0
ExecStart={ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd ; ignore_errors=no ; start_time=[Tue 2018-11-27 17:29:04 CST] ; stop_time=[n/a] ; pid=73827 ; code=(null) ; status=0/0 }
ExecReload={ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
Slice=system.slice
ControlGroup=/system.slice/docker.service
MemoryCurrent=1521336320
TasksCurrent=18446744073709551615
Delegate=yes
CPUAccounting=no
CPUShares=18446744073709551615
StartupCPUShares=18446744073709551615
CPUQuotaPerSecUSec=infinity
BlockIOAccounting=no
BlockIOWeight=18446744073709551615
StartupBlockIOWeight=18446744073709551615
MemoryAccounting=no
MemoryLimit=18446744073709551615
DevicePolicy=auto
TasksAccounting=no
TasksMax=18446744073709551615
UMask=0022
LimitCPU=18446744073709551615
LimitFSIZE=18446744073709551615
LimitDATA=18446744073709551615
LimitSTACK=18446744073709551615
LimitCORE=18446744073709551615
LimitRSS=18446744073709551615
LimitNOFILE=18446744073709551615
LimitAS=18446744073709551615
LimitNPROC=18446744073709551615
LimitMEMLOCK=65536
LimitLOCKS=18446744073709551615
LimitSIGPENDING=514808
LimitMSGQUEUE=819200
LimitNICE=0
LimitRTPRIO=0
LimitRTTIME=18446744073709551615
OOMScoreAdjust=0
Nice=0
IOScheduling=0
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SecureBits=0
CapabilityBoundingSet=18446744073709551615
AmbientCapabilities=0
MountFlags=0
PrivateTmp=no
PrivateNetwork=no
PrivateDevices=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=no
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=0
RuntimeDirectoryMode=0755
KillMode=process
KillSignal=15
SendSIGKILL=yes
SendSIGHUP=no
Id=docker.service
Names=docker.service
Requires=basic.target
Wants=network-online.target system.slice
Conflicts=shutdown.target
Before=shutdown.target
After=system.slice network-online.target basic.target firewalld.service systemd-journald.socket
Documentation=https://docs.docker.com
Description=Docker Application Container Engine
LoadState=loaded
ActiveState=active
SubState=running
FragmentPath=/usr/lib/systemd/system/docker.service
UnitFileState=disabled
UnitFilePreset=disabled
InactiveExitTimestamp=Tue 2018-11-27 17:29:04 CST
InactiveExitTimestampMonotonic=10194331578132
ActiveEnterTimestamp=Tue 2018-11-27 17:29:06 CST
ActiveEnterTimestampMonotonic=10194333691406
ActiveExitTimestamp=Tue 2018-11-27 17:29:03 CST
ActiveExitTimestampMonotonic=10194330560298
InactiveEnterTimestamp=Tue 2018-11-27 17:29:04 CST
InactiveEnterTimestampMonotonic=10194331577212
CanStart=yes
CanStop=yes
CanReload=yes
CanIsolate=no
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=no
IgnoreOnSnapshot=no
NeedDaemonReload=no
JobTimeoutUSec=0
JobTimeoutAction=none
ConditionResult=yes
AssertResult=yes
ConditionTimestamp=Tue 2018-11-27 17:29:04 CST
ConditionTimestampMonotonic=10194331577412
AssertTimestamp=Tue 2018-11-27 17:29:04 CST
AssertTimestampMonotonic=10194331577412
Transient=no

No kubectl

---

Packages

No dpkg Have rpm Output of "rpm -qa|egrep "(cc-oci-runtimecc-runtimerunv|kata-proxy|kata-runtime|kata-shim|kata-ksm-throttler|kata-containers-image|linux-container|qemu-)"":

kata-shim-bin-1.4.0+git.147ffb1-33.7.x86_64
kata-shim-1.4.0+git.147ffb1-33.7.x86_64
kata-proxy-bin-1.4.0+git.209cb5f-32.7.x86_64
kata-proxy-1.4.0+git.209cb5f-32.7.x86_64
kata-linux-container-4.14.67-19.7.x86_64
kata-ksm-throttler-1.1.0.git+1fecaff-12.7.x86_64
kata-containers-image-1.4.0-12.7.x86_64
kata-runtime-1.4.0+git.7b63f21-17.7.x86_64
linux-container-4.9.47-80.1.x86_64
qemu-vanilla-data-2.11.2+git.0982a56-3.7.x86_64
qemu-lite-data-2.11.0+git.f886228-37.7.x86_64
qemu-lite-bin-2.11.0+git.f886228-37.7.x86_64
qemu-lite-2.11.0+git.f886228-37.7.x86_64
qemu-vanilla-bin-2.11.2+git.0982a56-3.7.x86_64
qemu-vanilla-2.11.2+git.0982a56-3.7.x86_64

---

[jodh-intel]

Hi @linxiulei - thanks for providing that. The problem I think is that you are using an image and not an initrd in your configuration.toml.

@bergwolf - can you confirm this is a requirement for factory support? If so, we need to change the runtime to:

• Display a clear message if enable_template=true but initrd= is not set.
• Add a comment to configuration.toml.in explaining the requirement.

[jodh-intel]

@linxiulei - Can you confirm that uncommenting the initrd= entry in your configuration.toml and commenting out the image= entry works as expected?

See https://github.com/kata-containers/documentation/blob/master/Developer-Guide.md#configure-to-use-initrd-or-rootfs-image

[bergwolf]

@jodh-intel Yes, initrd is required to use vm templates.

[jodh-intel]

@bergwolf - ok - I'll fix the issues listed in https://github.com/kata-containers/runtime/issues/945#issuecomment-442420161 ...

[jodh-intel]

Fixed on #949.

@linxiulei - are you happy for us to close this issue?

[linxiulei]

Great : )