Closed GabyCT closed 6 years ago
This is the environment information
# Meta details
Running `kata-collect-data.sh` version `1.2.0 (commit 56ba8adc3ac30ba3d30d420773512a6f5ae4d521)` at `2018-09-10.19:23:12.580848865+0000`.
---
Runtime is `/usr/local/bin/kata-runtime`.
# `kata-env`
Output of "`/usr/local/bin/kata-runtime kata-env`":
[Meta]
Version = "1.0.15"
[Runtime]
Debug = false
Path = "/usr/local/bin/kata-runtime"
[Runtime.Version]
Semver = "1.2.0"
Commit = "56ba8adc3ac30ba3d30d420773512a6f5ae4d521"
OCI = "1.0.1"
[Runtime.Config]
Path = "/usr/share/defaults/kata-containers/configuration.toml"
[Hypervisor]
MachineType = "pc"
Version = "QEMU emulator version 2.11.0\nCopyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers"
Path = "/usr/bin/qemu-lite-system-x86_64"
BlockDeviceDriver = "virtio-scsi"
Msize9p = 8192
Debug = false
UseVSock = false
[Image]
Path = "/usr/share/kata-containers/kata-containers-image_clearlinux_1.2.0_agent_fcfa054a757.img"
[Kernel]
Path = "/usr/share/kata-containers/vmlinuz-4.14.67-11"
Parameters = "agent.log=debug"
[Initrd]
Path = ""
[Proxy]
Type = "kataProxy"
Version = "kata-proxy version 1.2.0-5377b5db51fe59b52b07ee442d3598802b23d7f7"
Path = "/usr/libexec/kata-containers/kata-proxy"
Debug = true
[Shim]
Type = "kataShim"
Version = "kata-shim version 1.2.0-35c7dbabc677b0d67b4b5b21fc87c90090bc5a82"
Path = "/usr/libexec/kata-containers/kata-shim"
Debug = true
[Agent]
Type = "kata"
[Host]
Kernel = "4.15.0-1022-azure"
Architecture = "amd64"
VMContainerCapable = true
SupportVSocks = false
[Host.Distro]
Name = "Ubuntu"
Version = "16.04"
[Host.CPU]
Vendor = "GenuineIntel"
Model = "Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz"
/etc/kata-containers/configuration.toml
/usr/share/defaults/kata-containers/configuration.toml
Config file /etc/kata-containers/configuration.toml
not found
Output of "cat "/usr/share/defaults/kata-containers/configuration.toml"
":
# Copyright (c) 2017-2018 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
#
# XXX: WARNING: this file is auto-generated.
# XXX:
# XXX: Source file: "cli/config/configuration.toml.in"
# XXX: Project:
# XXX: Name: Kata Containers
# XXX: Type: kata
[hypervisor.qemu]
path = "/usr/bin/qemu-lite-system-x86_64"
kernel = "/usr/share/kata-containers/vmlinuz.container"
image = "/usr/share/kata-containers/kata-containers.img"
machine_type = "pc"
# Optional space-separated list of options to pass to the guest kernel.
# For example, use `kernel_params = "vsyscall=emulate"` if you are having
# trouble running pre-2.15 glibc.
#
# WARNING: - any parameter specified here will take priority over the default
# parameter value of the same name used to start the virtual machine.
# Do not set values here unless you understand the impact of doing so as you
# may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a
# container and look for 'default-kernel-parameters' log entries.
kernel_params = " agent.log=debug"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = ""
# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
machine_accelerators=""
# Default number of vCPUs per SB/VM:
# unspecified or 0 --> will be set to 1
# < 0 --> will be set to the actual number of physical cores
# > 0 <= number of physical cores --> will be set to the specified number
# > number of physical cores --> will be set to the actual number of physical cores
default_vcpus = 1
# Default maximum number of vCPUs per SB/VM:
# unspecified or == 0 --> will be set to the actual number of physical cores or to the maximum number
# of vCPUs supported by KVM if that number is exceeded
# > 0 <= number of physical cores --> will be set to the specified number
# > number of physical cores --> will be set to the actual number of physical cores or to the maximum number
# of vCPUs supported by KVM if that number is exceeded
# WARNING: Depending of the architecture, the maximum number of vCPUs supported by KVM is used when
# the actual number of physical cores is greater than it.
# WARNING: Be aware that this value impacts the virtual machine's memory footprint and CPU
# the hotplug functionality. For example, `default_maxvcpus = 240` specifies that until 240 vCPUs
# can be added to a SB/VM, but the memory footprint will be big. Another example, with
# `default_maxvcpus = 8` the memory footprint will be small, but 8 will be the maximum number of
# vCPUs supported by the SB/VM. In general, we recommend that you do not edit this variable,
# unless you know what are you doing.
default_maxvcpus = 0
# Bridges can be used to hot plug devices.
# Limitations:
# * Currently only pci bridges are supported
# * Until 30 devices per bridge can be hot plugged.
# * Until 5 PCI bridges can be cold plugged per VM.
# This limitation could be a bug in qemu or in the kernel
# Default number of bridges per SB/VM:
# unspecified or 0 --> will be set to 1
# > 1 <= 5 --> will be set to the specified number
# > 5 --> will be set to 5
default_bridges = 1
# Default memory size in MiB for SB/VM.
# If unspecified then it will be set 2048 MiB.
#default_memory = 2048
# Disable block device from being used for a container's rootfs.
# In case of a storage driver like devicemapper where a container's
# root file system is backed by a block device, the block device is passed
# directly to the hypervisor for performance reasons.
# This flag prevents the block device from being passed to the hypervisor,
# 9pfs is used instead to pass the rootfs.
disable_block_device_use = false
# Block storage driver to be used for the hypervisor in case the container
# rootfs is backed by a block device. This is either virtio-scsi or
# virtio-blk.
block_device_driver = "virtio-scsi"
# Enable iothreads (data-plane) to be used. This causes IO to be
# handled in a separate IO thread. This is currently only implemented
# for SCSI.
#
enable_iothreads = false
# Enable pre allocation of VM RAM, default false
# Enabling this will result in lower container density
# as all of the memory will be allocated and locked
# This is useful when you want to reserve all the memory
# upfront or in the cases where you want memory latencies
# to be very predictable
# Default false
#enable_mem_prealloc = true
# Enable huge pages for VM RAM, default false
# Enabling this will result in the VM memory
# being allocated using huge pages.
# This is useful when you want to use vhost-user network
# stacks within the container. This will automatically
# result in memory pre allocation
#enable_hugepages = true
# Enable swap of vm memory. Default false.
# The behaviour is undefined if mem_prealloc is also set to true
#enable_swap = true
# This option changes the default hypervisor and kernel parameters
# to enable debug output where available. This extra output is added
# to the proxy logs, but only when proxy debug is also enabled.
#
# Default false
enable_debug = true
# Disable the customizations done in the runtime when it detects
# that it is running on top a VMM. This will result in the runtime
# behaving as it would when running on bare metal.
#
#disable_nesting_checks = true
# This is the msize used for 9p shares. It is the number of bytes
# used for 9p packet payload.
#msize_9p = 8192
# If true and vsocks are supported, use vsocks to communicate directly
# with the agent and no proxy is started, otherwise use unix
# sockets and start a proxy to communicate with the agent.
# Default false
#use_vsock = true
# VFIO devices are hotplugged on a bridge by default.
# Enable hotplugging on root bus. This may be required for devices with
# a large PCI bar, as this is a current limitation with hotplugging on
# a bridge. This value is valid for "pc" machine type.
# Default false
#hotplug_vfio_on_root_bus = true
[factory]
# VM templating support. Once enabled, new VMs are created from template
# using vm cloning. They will share the same initial kernel, initramfs and
# agent memory by mapping it readonly. It helps speeding up new container
# creation and saves a lot of memory if there are many kata containers running
# on the same host.
#
# When disabled, new VMs are created from scratch.
#
# Default false
#enable_template = true
[proxy.kata]
path = "/usr/libexec/kata-containers/kata-proxy"
# If enabled, proxy messages will be sent to the system log
# (default: disabled)
enable_debug = true
[shim.kata]
path = "/usr/libexec/kata-containers/kata-shim"
# If enabled, shim messages will be sent to the system log
# (default: disabled)
enable_debug = true
[agent.kata]
# There is no field for this section. The goal is only to be able to
# specify which type of agent the user wants to use.
[runtime]
# If enabled, the runtime will log additional debug messages to the
# system log
# (default: disabled)
enable_debug = true
#
# Internetworking model
# Determines how the VM should be connected to the
# the container network interface
# Options:
#
# - bridged
# Uses a linux bridge to interconnect the container interface to
# the VM. Works for most cases except macvlan and ipvlan.
#
# - macvtap
# Used when the Container network interface can be bridged using
# macvtap.
internetworking_model="macvtap"
# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)
#enable_tracing = true
---
osbuilder:
url: "https://github.com/kata-containers/osbuilder"
version: "unknown"
rootfs-creation-time: "2018-08-13T22:51:39.765008919+0000Z"
description: "osbuilder rootfs"
file-format-version: "0.0.2"
architecture: "x86_64"
base-distro:
name: "Clear"
version: "24400"
packages:
default:
- "iptables-bin"
- "libudev0-shim"
- "systemd"
extra:
agent:
url: "https://github.com/kata-containers/agent"
name: "kata-agent"
version: "1.2.0-fcfa054a757e7c17afba47b0b4d7e91cbb8688ed"
agent-is-init-daemon: "no"
No initrd
Recent runtime problems found in system journal:
time="2018-09-10T18:39:22.104269887Z" level=error msg="Container 99352baedebef72bda506ee2d7912fea139a5111bf52fea1441a166ec32295af not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=99352baedebef72bda506ee2d7912fea139a5111bf52fea1441a166ec32295af name=kata-runtime pid=31847 sandbox=99352baedebef72bda506ee2d7912fea139a5111bf52fea1441a166ec32295af source=runtime
time="2018-09-10T18:39:22.224667697Z" level=error msg="Container ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 name=kata-runtime pid=31879 sandbox=ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 source=runtime
time="2018-09-10T18:39:22.291984199Z" level=error msg="Container a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 name=kata-runtime pid=31907 sandbox=a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 source=runtime
time="2018-09-10T18:39:22.419915353Z" level=error msg="Container 5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 name=kata-runtime pid=31948 sandbox=5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 source=runtime
time="2018-09-10T18:39:22.474790247Z" level=error msg="Container a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 name=kata-runtime pid=31979 sandbox=a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 source=runtime
time="2018-09-10T18:39:22.501204552Z" level=error msg="Container 8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e name=kata-runtime pid=31981 sandbox=8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e source=runtime
time="2018-09-10T18:39:22.525116275Z" level=error msg="Container 4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe name=kata-runtime pid=31959 sandbox=4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe source=runtime
time="2018-09-10T18:39:22.639008633Z" level=error msg="Container 1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 name=kata-runtime pid=32067 sandbox=1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 source=runtime
time="2018-09-10T18:39:22.66378105Z" level=error msg="Container ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 name=kata-runtime pid=32062 sandbox=ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 source=runtime
time="2018-09-10T18:39:22.681165821Z" level=error msg="Container f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 name=kata-runtime pid=32035 sandbox=f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 source=runtime
time="2018-09-10T18:39:22.693837628Z" level=error msg="Container a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 name=kata-runtime pid=32137 sandbox=a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 source=runtime
time="2018-09-10T18:39:22.749993312Z" level=error msg="Container 1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 name=kata-runtime pid=32125 sandbox=1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 source=runtime
time="2018-09-10T18:39:22.761669326Z" level=error msg="Container a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 name=kata-runtime pid=32144 sandbox=a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 source=runtime
time="2018-09-10T18:39:22.821641883Z" level=error msg="Container 5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 name=kata-runtime pid=32184 sandbox=5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 source=runtime
time="2018-09-10T18:39:22.840467643Z" level=error msg="Container a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b name=kata-runtime pid=32215 sandbox=a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b source=runtime
time="2018-09-10T18:39:22.848854381Z" level=error msg="Container 1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 name=kata-runtime pid=32262 sandbox=1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 source=runtime
time="2018-09-10T18:39:22.860934992Z" level=error msg="Container 4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe name=kata-runtime pid=32280 sandbox=4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe source=runtime
time="2018-09-10T18:39:22.884499018Z" level=error msg="Container 8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e name=kata-runtime pid=32263 sandbox=8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e source=runtime
time="2018-09-10T18:39:22.883301927Z" level=error msg="Container 399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 name=kata-runtime pid=32301 sandbox=399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 source=runtime
time="2018-09-10T18:39:22.95994866Z" level=error msg="Container f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 name=kata-runtime pid=32352 sandbox=f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 source=runtime
time="2018-09-10T18:39:23.016377943Z" level=error msg="Container 1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 name=kata-runtime pid=32438 sandbox=1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 source=runtime
time="2018-09-10T18:39:23.022552297Z" level=error msg="Container a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b name=kata-runtime pid=32472 sandbox=a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b source=runtime
time="2018-09-10T18:39:23.066921469Z" level=error msg="Container 50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 name=kata-runtime pid=32490 sandbox=50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 source=runtime
time="2018-09-10T18:39:23.085217134Z" level=error msg="Container 399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 name=kata-runtime pid=32536 sandbox=399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 source=runtime
time="2018-09-10T18:39:23.122106461Z" level=error msg="Container 50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 name=kata-runtime pid=32570 sandbox=50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 source=runtime
time="2018-09-10T18:41:14.968235928Z" level=debug msg="Could not retrieve anything from storage" arch=amd64 command=create container=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 name=kata-runtime pid=1824 source=virtcontainers subsystem=kata_agent
time="2018-09-10T18:41:14.968307627Z" level=warning msg="fetch sandbox device failed" arch=amd64 command=create container=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 error="open /run/vc/sbs/ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81/devices.json: no such file or directory" name=kata-runtime pid=1824 sandbox=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 sandboxid=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 source=virtcontainers subsystem=sandbox
time="2018-09-10T18:41:14.970363412Z" level=debug arch=amd64 command=create container=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 debug systemd.show_status=true systemd.log_level=debug" name=kata-runtime pid=1824 source=virtcontainers subsystem=qemu
time="2018-09-10T18:52:29.963265499Z" level=error msg="Container not ready, running or paused, impossible to signal the container" arch=amd64 command=kill container=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 name=kata-runtime pid=2644 sandbox=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 source=runtime
time="2018-09-10T18:52:30.011091823Z" level=error msg="Container ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 name=kata-runtime pid=2679 sandbox=ac60762c6983ef872e1df96dda8bc193d65d059beea502614a9516985d799b81 source=runtime
time="2018-09-10T18:54:26.696272054Z" level=debug msg="Could not retrieve anything from storage" arch=amd64 command=create container=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 name=kata-runtime pid=3089 source=virtcontainers subsystem=kata_agent
time="2018-09-10T18:54:26.696362054Z" level=warning msg="fetch sandbox device failed" arch=amd64 command=create container=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 error="open /run/vc/sbs/0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7/devices.json: no such file or directory" name=kata-runtime pid=3089 sandbox=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 sandboxid=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 source=virtcontainers subsystem=sandbox
time="2018-09-10T18:54:26.698738435Z" level=debug arch=amd64 command=create container=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 debug systemd.show_status=true systemd.log_level=debug" name=kata-runtime pid=3089 source=virtcontainers subsystem=qemu
time="2018-09-10T19:08:52.243393682Z" level=error msg="Container not ready, running or paused, impossible to signal the container" arch=amd64 command=kill container=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 name=kata-runtime pid=4418 sandbox=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 source=runtime
time="2018-09-10T19:08:52.293968744Z" level=error msg="Container 0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 name=kata-runtime pid=4451 sandbox=0fd6dccc2181d46d6bf5a2cb01d61ca2a81554e0dbd197a95fa22dbbdd4dcae7 source=runtime
time="2018-09-10T19:09:39.528163725Z" level=debug msg="Could not retrieve anything from storage" arch=amd64 command=create container=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f name=kata-runtime pid=4797 source=virtcontainers subsystem=kata_agent
time="2018-09-10T19:09:39.528232824Z" level=warning msg="fetch sandbox device failed" arch=amd64 command=create container=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f error="open /run/vc/sbs/222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f/devices.json: no such file or directory" name=kata-runtime pid=4797 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f sandboxid=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=virtcontainers subsystem=sandbox
time="2018-09-10T19:09:39.530282807Z" level=debug arch=amd64 command=create container=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 debug systemd.show_status=true systemd.log_level=debug" name=kata-runtime pid=4797 source=virtcontainers subsystem=qemu
time="2018-09-10T19:09:45.993448207Z" level=error msg="Container not ready, running or paused, impossible to signal the container" arch=amd64 command=kill container=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f name=kata-runtime pid=4914 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=runtime
time="2018-09-10T19:09:46.038094923Z" level=error msg="Container 222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f name=kata-runtime pid=4945 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=runtime
time="2018-09-10T19:09:58.314159838Z" level=debug msg="Could not retrieve anything from storage" arch=amd64 command=create container=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 name=kata-runtime pid=5251 source=virtcontainers subsystem=kata_agent
time="2018-09-10T19:09:58.314327937Z" level=warning msg="fetch sandbox device failed" arch=amd64 command=create container=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 error="open /run/vc/sbs/c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624/devices.json: no such file or directory" name=kata-runtime pid=5251 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 sandboxid=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=virtcontainers subsystem=sandbox
time="2018-09-10T19:09:58.317054513Z" level=debug arch=amd64 command=create container=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 debug systemd.show_status=true systemd.log_level=debug" name=kata-runtime pid=5251 source=virtcontainers subsystem=qemu
time="2018-09-10T19:10:03.56219619Z" level=error msg="Container not ready, running or paused, impossible to signal the container" arch=amd64 command=kill container=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 name=kata-runtime pid=5370 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=runtime
time="2018-09-10T19:10:03.618518207Z" level=error msg="Container c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 name=kata-runtime pid=5408 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=runtime
time="2018-09-10T19:10:15.529930066Z" level=debug msg="Could not retrieve anything from storage" arch=amd64 command=create container=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 name=kata-runtime pid=5703 source=virtcontainers subsystem=kata_agent
time="2018-09-10T19:10:15.529995266Z" level=warning msg="fetch sandbox device failed" arch=amd64 command=create container=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 error="open /run/vc/sbs/0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2/devices.json: no such file or directory" name=kata-runtime pid=5703 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 sandboxid=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=virtcontainers subsystem=sandbox
time="2018-09-10T19:10:15.532214347Z" level=debug arch=amd64 command=create container=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 debug systemd.show_status=true systemd.log_level=debug" name=kata-runtime pid=5703 source=virtcontainers subsystem=qemu
time="2018-09-10T19:10:46.51121865Z" level=error msg="Container not ready, running or paused, impossible to signal the container" arch=amd64 command=kill container=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 name=kata-runtime pid=5841 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=runtime
time="2018-09-10T19:10:46.559960634Z" level=error msg="Container 0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 not ready, running or paused, cannot send a signal" arch=amd64 command=kill container=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 name=kata-runtime pid=5875 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=runtime
Recent proxy problems found in system journal:
time="2018-09-10T19:09:44.704144895Z" level=info msg="time=\"2018-09-10T19:09:44.687634407Z\" level=debug msg=\"request end\" duration=3.054519126s error=EOF name=kata-agent pid=103 request=/grpc.AgentService/ReadStdout resp= sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent\n" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent
time="2018-09-10T19:09:44.706499575Z" level=info msg="time=\"2018-09-10T19:09:44.689866569Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=103 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f signal=\"child exited\" source=agent\n" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent
time="2018-09-10T19:09:44.722484937Z" level=info msg="time=\"2018-09-10T19:09:44.705960855Z\" level=debug msg=\"request end\" duration=\"384.234µs\" name=kata-agent pid=103 request=/grpc.AgentService/SignalProcess resp=\"&Empty{}\" sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent\n" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent
time="2018-09-10T19:09:44.726336704Z" level=info msg="time=\"2018-09-10T19:09:44.709680653Z\" level=debug msg=\"request end\" duration=\"5.708µs\" error=\"rpc error: code = NotFound desc = Process 222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f not found (container 222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f)\" name=kata-agent pid=103 request=/grpc.AgentService/WaitProcess resp= sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent\n" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent
time="2018-09-10T19:09:45.935179908Z" level=info msg="time=\"2018-09-10T19:09:45.918568357Z\" level=debug msg=\"request end\" duration=1.205598344s name=kata-agent pid=103 request=/grpc.AgentService/RemoveContainer resp=\"&Empty{}\" sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent\n" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent
time="2018-09-10T19:09:45.99545739Z" level=info msg="[ 5.406298] sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x04 driverbyte=0x00\n" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent
time="2018-09-10T19:09:46.06755737Z" level=info msg="time=\"2018-09-10T19:09:46.050904845Z\" level=debug msg=\"request end\" duration=11.766217ms name=kata-agent pid=103 request=/grpc.AgentService/DestroySandbox resp=\"&Empty{}\" sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent\n" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=agent
time="2018-09-10T19:09:46.069381854Z" level=fatal msg="failed to handle exit signal" error="close unix @->/run/vc/vm/222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f/kata.sock: use of closed network connection" name=kata-proxy pid=4843 sandbox=222f03307f60e58c5a301229067bcbdccf7770b216a94b1b537594cfaab5bb6f source=proxy
time="2018-09-10T19:09:59.711880638Z" level=info msg="[ 0.279608] EXT4-fs (pmem0p1): DAX enabled. Warning: EXPERIMENTAL, use at your own risk\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:09:59.713250126Z" level=info msg="[ 0.280726] EXT4-fs (pmem0p1): mounted filesystem with ordered data mode. Opts: dax,data=ordered,errors=remount-ro\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:09:59.762982199Z" level=info msg="[ 0.330554] systemd-gpt-auto-generator[68]: Failed to chase block device '/', ignoring: No such file or directory\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:09:59.807436317Z" level=info msg="[ 0.375129] systemd[70]: tmp.mount: Failed to execute command: No such file or directory\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:09:59.817028735Z" level=info msg="[ 0.384595] systemd-journald[73]: Failed to open configuration file '/etc/systemd/journald.conf': No such file or directory\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:09:59.83854235Z" level=info msg="[\x1b[0;1;31mFAILED\x1b[0m] Failed to mount Temporary Directory (/tmp).\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:09:59.840798531Z" level=info msg="[\x1b[0;1;33mDEPEND\x1b[0m] Dependency failed for Network Time Synchronization.\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:00.144196426Z" level=info msg="time=\"2018-09-10T19:10:00.126292803Z\" level=debug msg=\"request end\" duration=13.23638ms name=kata-agent pid=107 request=/grpc.AgentService/CreateSandbox resp=\"&Empty{}\" sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:00.452787277Z" level=info msg="time=\"2018-09-10T19:10:00.434837784Z\" level=debug msg=\"request end\" duration=268.102896ms name=kata-agent pid=107 request=/grpc.AgentService/CreateContainer resp=\"&Empty{}\" sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:00.454218365Z" level=info msg="time=\"2018-09-10T19:10:00.435415578Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=107 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 signal=\"child exited\" source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:00.47229531Z" level=info msg="time=\"2018-09-10T19:10:00.454296865Z\" level=debug msg=\"request end\" duration=\"189.341µs\" name=kata-agent pid=107 request=/grpc.AgentService/CloseStdin resp=\"&Empty{}\" sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:00.505673323Z" level=info msg="time=\"2018-09-10T19:10:00.48758591Z\" level=debug msg=\"request end\" duration=4.367948ms name=kata-agent pid=107 request=/grpc.AgentService/StartContainer resp=\"&Empty{}\" sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:02.501463593Z" level=info msg="time=\"2018-09-10T19:10:02.483577745Z\" level=debug msg=\"request end\" duration=8.925939ms error=EOF name=kata-agent pid=107 request=/grpc.AgentService/ReadStderr resp= sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:02.502159787Z" level=info msg="time=\"2018-09-10T19:10:02.484136447Z\" level=debug msg=\"request end\" duration=2.031612676s error=EOF name=kata-agent pid=107 request=/grpc.AgentService/ReadStdout resp= sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:02.503264878Z" level=info msg="time=\"2018-09-10T19:10:02.485124411Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=107 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 signal=\"child exited\" source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:02.520671028Z" level=info msg="time=\"2018-09-10T19:10:02.502740705Z\" level=debug msg=\"request end\" duration=\"277.055µs\" name=kata-agent pid=107 request=/grpc.AgentService/SignalProcess resp=\"&Empty{}\" sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:02.523636103Z" level=info msg="time=\"2018-09-10T19:10:02.505623358Z\" level=debug msg=\"request end\" duration=\"7.01µs\" error=\"rpc error: code = NotFound desc = Process c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 not found (container c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624)\" name=kata-agent pid=107 request=/grpc.AgentService/WaitProcess resp= sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:03.498653135Z" level=info msg="time=\"2018-09-10T19:10:03.480568186Z\" level=debug msg=\"request end\" duration=971.560368ms name=kata-agent pid=107 request=/grpc.AgentService/RemoveContainer resp=\"&Empty{}\" sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:03.561697795Z" level=info msg="[ 4.129242] sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x04 driverbyte=0x00\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:03.647878355Z" level=info msg="time=\"2018-09-10T19:10:03.629879635Z\" level=debug msg=\"request end\" duration=11.039188ms name=kata-agent pid=107 request=/grpc.AgentService/DestroySandbox resp=\"&Empty{}\" sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent\n" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=agent
time="2018-09-10T19:10:03.649024845Z" level=fatal msg="failed to handle exit signal" error="close unix @->/run/vc/vm/c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624/kata.sock: use of closed network connection" name=kata-proxy pid=5299 sandbox=c8b022d7aeb7f2e97e5a38859f552e2d91bb5d9ec98abca33a3e15194cc58624 source=proxy
time="2018-09-10T19:10:16.903634896Z" level=info msg="[ 0.289379] EXT4-fs (pmem0p1): DAX enabled. Warning: EXPERIMENTAL, use at your own risk\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:16.904971184Z" level=info msg="[ 0.290627] EXT4-fs (pmem0p1): mounted filesystem with ordered data mode. Opts: dax,data=ordered,errors=remount-ro\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:16.950261196Z" level=info msg="[ 0.335853] systemd-gpt-auto-generator[68]: Failed to chase block device '/', ignoring: No such file or directory\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:16.999277576Z" level=info msg="[ 0.384897] systemd-journald[70]: Failed to open configuration file '/etc/systemd/journald.conf': No such file or directory\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.023820466Z" level=info msg="[ 0.409504] systemd[77]: tmp.mount: Failed to execute command: No such file or directory\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.053183614Z" level=info msg="[\x1b[0;1;31mFAILED\x1b[0m] Failed to mount Temporary Directory (/tmp).\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.054334904Z" level=info msg="[\x1b[0;1;33mDEPEND\x1b[0m] Dependency failed for Network Time Synchronization.\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.371874484Z" level=info msg="time=\"2018-09-10T19:10:17.353722195Z\" level=debug msg=\"request end\" duration=12.287739ms name=kata-agent pid=111 request=/grpc.AgentService/CreateSandbox resp=\"&Empty{}\" sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.677378267Z" level=info msg="time=\"2018-09-10T19:10:17.65903783Z\" level=debug msg=\"request end\" duration=259.629725ms name=kata-agent pid=111 request=/grpc.AgentService/CreateContainer resp=\"&Empty{}\" sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.681144834Z" level=info msg="time=\"2018-09-10T19:10:17.660322835Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=111 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 signal=\"child exited\" source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.695267913Z" level=info msg="time=\"2018-09-10T19:10:17.677032804Z\" level=debug msg=\"request end\" duration=\"12.856µs\" name=kata-agent pid=111 request=/grpc.AgentService/CloseStdin resp=\"&Empty{}\" sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:17.727463338Z" level=info msg="time=\"2018-09-10T19:10:17.709409123Z\" level=debug msg=\"request end\" duration=4.807998ms name=kata-agent pid=111 request=/grpc.AgentService/StartContainer resp=\"&Empty{}\" sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:44.022348306Z" level=info msg="time=\"2018-09-10T19:10:44.004308043Z\" level=debug msg=\"request end\" duration=9.268528ms error=EOF name=kata-agent pid=111 request=/grpc.AgentService/ReadStderr resp= sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:44.0230464Z" level=info msg="time=\"2018-09-10T19:10:44.004582262Z\" level=debug msg=\"request end\" duration=10.993114369s error=EOF name=kata-agent pid=111 request=/grpc.AgentService/ReadStdout resp= sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:44.024447988Z" level=info msg="time=\"2018-09-10T19:10:44.006326439Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=111 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 signal=\"child exited\" source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:44.042634533Z" level=info msg="time=\"2018-09-10T19:10:44.024435444Z\" level=debug msg=\"request end\" duration=\"505.333µs\" name=kata-agent pid=111 request=/grpc.AgentService/SignalProcess resp=\"&Empty{}\" sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:44.045939605Z" level=info msg="time=\"2018-09-10T19:10:44.027422036Z\" level=debug msg=\"request end\" duration=\"4.591µs\" error=\"rpc error: code = NotFound desc = Process 0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 not found (container 0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2)\" name=kata-agent pid=111 request=/grpc.AgentService/WaitProcess resp= sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:46.433815011Z" level=info msg="time=\"2018-09-10T19:10:46.415472555Z\" level=debug msg=\"request end\" duration=2.384999234s name=kata-agent pid=111 request=/grpc.AgentService/RemoveContainer resp=\"&Empty{}\" sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:46.508311175Z" level=info msg="[ 29.893538] sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x04 driverbyte=0x00\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:46.584819521Z" level=info msg="time=\"2018-09-10T19:10:46.566697597Z\" level=debug msg=\"request end\" duration=5.49602ms name=kata-agent pid=111 request=/grpc.AgentService/DestroySandbox resp=\"&Empty{}\" sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent\n" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=agent
time="2018-09-10T19:10:46.586012011Z" level=fatal msg="failed to handle exit signal" error="close unix @->/run/vc/vm/0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2/kata.sock: use of closed network connection" name=kata-proxy pid=5751 sandbox=0b8250d81b28265ce912d367b5464e2ea770033e1a155e90db4dd1c61c373af2 source=proxy
Recent shim problems found in system journal:
time="2018-09-10T18:10:37.992277139Z" level=info msg="copy stdout failed" container=20933f5cedba518522703a95b37e5072f8b48f3f50fede3c794e70b02cc12cba error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=20933f5cedba518522703a95b37e5072f8b48f3f50fede3c794e70b02cc12cba name=kata-shim pid=1 source=shim
time="2018-09-10T18:10:38.027588451Z" level=info msg="copy stdout failed" container=66fb45495410f248c62bbd9d38006726608c857ff4b78d5d041f3c32bee8ba09 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=66fb45495410f248c62bbd9d38006726608c857ff4b78d5d041f3c32bee8ba09 name=kata-shim pid=1 source=shim
time="2018-09-10T18:10:38.034007399Z" level=info msg="copy stdout failed" container=a2a2ae4ad4781c8e6f9c28a82a7a55bf54e6f3b1794f00edd53fa442fbbc2b21 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=a2a2ae4ad4781c8e6f9c28a82a7a55bf54e6f3b1794f00edd53fa442fbbc2b21 name=kata-shim pid=1 source=shim
time="2018-09-10T18:10:38.035909983Z" level=info msg="copy stdout failed" container=bbd8e265069f1f1e7c751401adf4a8dc780efed667c958299c8620e3164b0852 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=bbd8e265069f1f1e7c751401adf4a8dc780efed667c958299c8620e3164b0852 name=kata-shim pid=1 source=shim
time="2018-09-10T18:10:38.035470687Z" level=info msg="copy stdout failed" container=89423ec4684feb05c80d35f3a7671d2de3411384474d185ef04cd1ee4c42ba5d error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=89423ec4684feb05c80d35f3a7671d2de3411384474d185ef04cd1ee4c42ba5d name=kata-shim pid=1 source=shim
time="2018-09-10T18:10:38.056970811Z" level=info msg="copy stdout failed" container=b18e80df9ee05e60822559bcea6f7a72d307b24bf499628194a6d963af083b35 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=b18e80df9ee05e60822559bcea6f7a72d307b24bf499628194a6d963af083b35 name=kata-shim pid=1 source=shim
time="2018-09-10T18:10:38.125144255Z" level=info msg="copy stdout failed" container=17bf7e6fc50754168f9f5aa210b7041f2f42b224959af5166a185e7912d23781 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=17bf7e6fc50754168f9f5aa210b7041f2f42b224959af5166a185e7912d23781 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:10.531380562Z" level=info msg="copy stdout failed" container=05226c2279e00b7515f8740c499ad8e68966c456dd062864b2ea9771d6c3f7a1 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=05226c2279e00b7515f8740c499ad8e68966c456dd062864b2ea9771d6c3f7a1 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:10.616601264Z" level=info msg="copy stdout failed" container=64933cbb4a8011534a7a1a0d8a795aef155f2b877c99b058d705b7d8c3bcb2b2 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=64933cbb4a8011534a7a1a0d8a795aef155f2b877c99b058d705b7d8c3bcb2b2 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:10.912467042Z" level=info msg="copy stdout failed" container=333dd3d4b0d4ccd29e4887afd332689970a7f7b7ad896410d9d0aebba5048c7d error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=333dd3d4b0d4ccd29e4887afd332689970a7f7b7ad896410d9d0aebba5048c7d name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:10.952131417Z" level=info msg="copy stdout failed" container=17f182fff83fcfc15c37d8a8ab7ebd8c7621698800640754018c0e0925a34308 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=17f182fff83fcfc15c37d8a8ab7ebd8c7621698800640754018c0e0925a34308 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:10.964123719Z" level=info msg="copy stdout failed" container=944c140c43dfa05d71f2357bd07947cb20268c0d9340b94818d46202b1c2d6b5 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=944c140c43dfa05d71f2357bd07947cb20268c0d9340b94818d46202b1c2d6b5 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.01900137Z" level=info msg="copy stdout failed" container=85b616bfdeb97889aa338658dfa2006ef514d428f1ab9a308c6c5089a1d4f94b error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=85b616bfdeb97889aa338658dfa2006ef514d428f1ab9a308c6c5089a1d4f94b name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.191643457Z" level=info msg="copy stdout failed" container=9bab3d5478329f34c44c984cc87d93cfc4c5932e4ffc59f8cea303df8801c3ff error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=9bab3d5478329f34c44c984cc87d93cfc4c5932e4ffc59f8cea303df8801c3ff name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.208984715Z" level=info msg="copy stdout failed" container=def6bb57a181cbd71c2977cab57ae153021cb63039239c5fdc33d63cd2cfdb43 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=def6bb57a181cbd71c2977cab57ae153021cb63039239c5fdc33d63cd2cfdb43 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.284870494Z" level=info msg="copy stdout failed" container=9dc84e8a3d508e773a0df4f001661d602f54bf1675ac08ddec2c35a6f1d04cdd error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=9dc84e8a3d508e773a0df4f001661d602f54bf1675ac08ddec2c35a6f1d04cdd name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.355837613Z" level=info msg="copy stdout failed" container=68cd09fcae6588f5ae5d2ddbe9b0057f4fb92a4e82a97239b401994c0904f016 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=68cd09fcae6588f5ae5d2ddbe9b0057f4fb92a4e82a97239b401994c0904f016 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.365106737Z" level=info msg="copy stdout failed" container=b30636d25e2cf70532014335d3668aa17f58dd9b72abb1f5a8c914f117d87e44 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=b30636d25e2cf70532014335d3668aa17f58dd9b72abb1f5a8c914f117d87e44 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.494129181Z" level=info msg="copy stdout failed" container=2aefcc42e29f009b28c4f171780f81eb898501b36853f5cc71de18ad616fbc42 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=2aefcc42e29f009b28c4f171780f81eb898501b36853f5cc71de18ad616fbc42 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.499283838Z" level=info msg="copy stdout failed" container=0d2b518c25293e923741b216e0af2d25c2c9c6044cbf258797022da743ba0137 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=0d2b518c25293e923741b216e0af2d25c2c9c6044cbf258797022da743ba0137 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.527803505Z" level=info msg="copy stdout failed" container=1a95bd86b7aba991e758ca66ffcbca7ad8b7c2f0847c8cd70ee87cffdd237fc0 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=1a95bd86b7aba991e758ca66ffcbca7ad8b7c2f0847c8cd70ee87cffdd237fc0 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.529382392Z" level=info msg="copy stdout failed" container=94308685b22314e87728cc03f82bd4feb55c7e83a17be007efe028fd17af2fad error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=94308685b22314e87728cc03f82bd4feb55c7e83a17be007efe028fd17af2fad name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.629423873Z" level=info msg="copy stdout failed" container=09a7665817bba29bd26bea9c4283c2f41c41b9ac2ef68d46726baa30aebc9ea5 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=09a7665817bba29bd26bea9c4283c2f41c41b9ac2ef68d46726baa30aebc9ea5 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.706278144Z" level=info msg="copy stdout failed" container=73f8910399ded2a65dfb3a53899a7b9b21e363638a3e4d58e93c8981b4f0960b error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=73f8910399ded2a65dfb3a53899a7b9b21e363638a3e4d58e93c8981b4f0960b name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.735084408Z" level=info msg="copy stdout failed" container=b082fadcc8dc1951f7739f22877523bafc8fbb4e031bb883b5bb2a6078f6a7b7 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=b082fadcc8dc1951f7739f22877523bafc8fbb4e031bb883b5bb2a6078f6a7b7 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.750380883Z" level=info msg="copy stdout failed" container=68368f9863cee026a0842db7981c6fd85f2dc454c6ea805fb3941ab4d5d129c8 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=68368f9863cee026a0842db7981c6fd85f2dc454c6ea805fb3941ab4d5d129c8 name=kata-shim pid=1 source=shim
time="2018-09-10T18:24:11.805814329Z" level=info msg="copy stdout failed" container=aab8bb08352b3afd5c8df86fe0b24329a4e38e3a3681aa4121a40347fe043538 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=aab8bb08352b3afd5c8df86fe0b24329a4e38e3a3681aa4121a40347fe043538 name=kata-shim pid=1 source=shim
time="2018-09-10T18:26:47.561966381Z" level=info msg="copy stdout failed" container=3022c4c19583326201f0dddcc8cfc66eb25280b0563e2f35131895cb10294a3f error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=3022c4c19583326201f0dddcc8cfc66eb25280b0563e2f35131895cb10294a3f name=kata-shim pid=1 source=shim
time="2018-09-10T18:26:47.599455974Z" level=info msg="copy stdout failed" container=58bdeb4a19e79979a6bb3f3e34361d48e534cc033cdc83ffd5cbc4dee1a9f68c error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=58bdeb4a19e79979a6bb3f3e34361d48e534cc033cdc83ffd5cbc4dee1a9f68c name=kata-shim pid=1 source=shim
time="2018-09-10T18:26:47.611098078Z" level=info msg="copy stdout failed" container=322da4173eabfbc1cd2822077b55ca1ab07580afda6821aef07768e8d2badf32 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=322da4173eabfbc1cd2822077b55ca1ab07580afda6821aef07768e8d2badf32 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.176388982Z" level=info msg="copy stdout failed" container=468b7f82241170290e4e43de3734e6e77360472d4ab83b4277ca7fd1842a520d error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=468b7f82241170290e4e43de3734e6e77360472d4ab83b4277ca7fd1842a520d name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.201060199Z" level=info msg="copy stdout failed" container=1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=1c0f7727d8030704d68bc56b7499a888ef8e44ebdb4c8a9208e652f0ce8fb6e4 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.205295668Z" level=info msg="copy stdout failed" container=8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=8631d5297749b8d075fb5d08b61333fca41138fa5b7d86dfdc5d625dc166241e name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.216483585Z" level=info msg="copy stdout failed" container=872f0bbe0674f33ef5fde7a9b28c51d215e11b7c825ff3b6dbf3e27ef523973d error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=872f0bbe0674f33ef5fde7a9b28c51d215e11b7c825ff3b6dbf3e27ef523973d name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.230269584Z" level=info msg="copy stdout failed" container=047ca2c15a05fc7dd5302e53a60d2812276ad1e46de41da2e94a78af3e3b62a7 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=047ca2c15a05fc7dd5302e53a60d2812276ad1e46de41da2e94a78af3e3b62a7 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.25370451Z" level=info msg="copy stdout failed" container=7964cbfa0f0e6cfef411ba467d22e42a50a668ed6beacbc45743bc812d46cf0b error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=7964cbfa0f0e6cfef411ba467d22e42a50a668ed6beacbc45743bc812d46cf0b name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.259479568Z" level=info msg="copy stdout failed" container=5353b528f2b946306963177e5c9dd4872521a82eeb7260e767faf8615b68e790 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=5353b528f2b946306963177e5c9dd4872521a82eeb7260e767faf8615b68e790 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.262153548Z" level=info msg="copy stdout failed" container=99352baedebef72bda506ee2d7912fea139a5111bf52fea1441a166ec32295af error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=99352baedebef72bda506ee2d7912fea139a5111bf52fea1441a166ec32295af name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.265799721Z" level=info msg="copy stdout failed" container=a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=a7654eca7baf1818b68a55c8d7fc7f0591f3b001e93dbea79d71ac1c3b59311b name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.273613463Z" level=info msg="copy stdout failed" container=f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=f4b0f44d85be981d880e2efc1f8833b486179b8829e1598d3aa88dbd18e653e9 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.313017272Z" level=info msg="copy stdout failed" container=1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=1b580263da5fe29fc5aa0c218532f58d9a009088990ad264e3064f7da4541143 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.31600915Z" level=info msg="copy stdout failed" container=5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=5fa9114a634f55f480c57d5d0827107015f306e80457b2e994909f1d6d9e9555 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.327698764Z" level=info msg="copy stdout failed" container=86831e72c9361d69a64c3db62914babe6cd55a1e03ee26fc86c3182a6999dbc0 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=86831e72c9361d69a64c3db62914babe6cd55a1e03ee26fc86c3182a6999dbc0 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.341876359Z" level=info msg="copy stdout failed" container=399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=399b058383ad64966d1d71d0bd318f5dc2ec64fd129f1ea76ec0825252f5cc94 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.356259953Z" level=info msg="copy stdout failed" container=ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=ee83b19347f2d69547c3aaaee7924e30fbd22d2127c287284a705573e20d74d8 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.362827604Z" level=info msg="copy stdout failed" container=4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=4e4a46f060caeea72d5b823935a33b7a5b097a4e5cb541fd84c09a53df2327fe name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.397149951Z" level=info msg="copy stdout failed" container=a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=a915056b7b62bca1b25595a31abd3ffd6cc8cc44f9eba03f3a1647f4aa72c7e3 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.402915108Z" level=info msg="copy stdout failed" container=536b9faa54481f93b59e70d0ca5e109ab0d60d206ab8f9526f94701071ac2662 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=536b9faa54481f93b59e70d0ca5e109ab0d60d206ab8f9526f94701071ac2662 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.505042654Z" level=info msg="copy stdout failed" container=50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=50e44b75b8a85b6cb691c1e1a75286ca00b65809b420ab8a899bc89ccd7ba4a7 name=kata-shim pid=1 source=shim
time="2018-09-10T18:39:15.510619813Z" level=info msg="copy stdout failed" container=a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 error="rpc error: code = Unknown desc = read /dev/ptmx: input/output error" exec-id=a6d017c70007fcbc978a4593f43b9bdd20780920bdf6db7f45d97a667db9d851 name=kata-shim pid=1 source=shim
Have docker
Output of "docker version
":
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:24:56 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:23:21 2018
OS/Arch: linux/amd64
Experimental: false
Output of "docker info
":
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 36
Server Version: 18.06.1-ce
Storage Driver: devicemapper
Pool Name: docker-8:1-770219-pool
Pool Blocksize: 65.54kB
Base Device Size: 10.74GB
Backing Filesystem: xfs
Udev Sync Supported: true
Data file: /dev/loop0
Metadata file: /dev/loop1
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Data Space Used: 5.24GB
Data Space Total: 107.4GB
Data Space Available: 18.56GB
Metadata Space Used: 6.345MB
Metadata Space Total: 2.147GB
Metadata Space Available: 2.141GB
Thin Pool Minimum Free Space: 10.74GB
Deferred Removal Enabled: true
Deferred Deletion Enabled: true
Deferred Deleted Device Count: 0
Library Version: 1.02.110 (2015-10-30)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: kata-runtime runc
Default Runtime: kata-runtime
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 56ba8adc3ac30ba3d30d420773512a6f5ae4d521 (expected: 69663f0bd4b60df09991c08812a60108003fa340)
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-1022-azure
Operating System: Ubuntu 16.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.67GiB
Name: gabycrio
ID: FZLU:4U5U:LHMO:JWLM:PDQY:7KJF:TZAK:I4NQ:SPQ7:VJI2:BPVA:J7UW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 24
Goroutines: 45
System Time: 2018-09-10T19:23:16.248383489Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
WARNING: No swap limit support
Output of "systemctl show docker
":
Type=notify
Restart=on-failure
NotifyAccess=main
RestartUSec=100ms
TimeoutStartUSec=infinity
TimeoutStopUSec=1min 30s
RuntimeMaxUSec=infinity
WatchdogUSec=0
WatchdogTimestamp=Mon 2018-09-10 18:03:54 UTC
WatchdogTimestampMonotonic=336286404354
FailureAction=none
PermissionsStartOnly=no
RootDirectoryStartOnly=no
RemainAfterExit=no
GuessMainPID=yes
MainPID=17850
ControlPID=0
FileDescriptorStoreMax=0
NFileDescriptorStore=0
StatusErrno=0
Result=success
ExecMainStartTimestamp=Mon 2018-09-10 18:03:53 UTC
ExecMainStartTimestampMonotonic=336285716539
ExecMainExitTimestampMonotonic=0
ExecMainPID=17850
ExecMainCode=0
ExecMainStatus=0
ExecStart={ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -D --add-runtime kata-runtime=/usr/local/bin/kata-runtime --default-runtime=kata-runtime ; ignore_errors=no ; start_time=[Mon 2018-09-10 18:03:53 UTC] ; stop_time=[n/a] ; pid=17850 ; code=(null) ; status=0/0 }
ExecReload={ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
Slice=system.slice
ControlGroup=/system.slice/docker.service
MemoryCurrent=224915456
CPUUsageNSec=3805057772678
TasksCurrent=76
Delegate=yes
CPUAccounting=no
CPUShares=18446744073709551615
StartupCPUShares=18446744073709551615
CPUQuotaPerSecUSec=infinity
BlockIOAccounting=no
BlockIOWeight=18446744073709551615
StartupBlockIOWeight=18446744073709551615
MemoryAccounting=no
MemoryLimit=18446744073709551615
DevicePolicy=auto
TasksAccounting=no
TasksMax=18446744073709551615
UMask=0022
LimitCPU=18446744073709551615
LimitCPUSoft=18446744073709551615
LimitFSIZE=18446744073709551615
LimitFSIZESoft=18446744073709551615
LimitDATA=18446744073709551615
LimitDATASoft=18446744073709551615
LimitSTACK=18446744073709551615
LimitSTACKSoft=8388608
LimitCORE=18446744073709551615
LimitCORESoft=18446744073709551615
LimitRSS=18446744073709551615
LimitRSSSoft=18446744073709551615
LimitNOFILE=1048576
LimitNOFILESoft=1048576
LimitAS=18446744073709551615
LimitASSoft=18446744073709551615
LimitNPROC=18446744073709551615
LimitNPROCSoft=18446744073709551615
LimitMEMLOCK=65536
LimitMEMLOCKSoft=65536
LimitLOCKS=18446744073709551615
LimitLOCKSSoft=18446744073709551615
LimitSIGPENDING=64083
LimitSIGPENDINGSoft=64083
LimitMSGQUEUE=819200
LimitMSGQUEUESoft=819200
LimitNICE=0
LimitNICESoft=0
LimitRTPRIO=0
LimitRTPRIOSoft=0
LimitRTTIME=18446744073709551615
LimitRTTIMESoft=18446744073709551615
OOMScoreAdjust=0
Nice=0
IOScheduling=0
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SyslogLevel=6
SyslogFacility=3
SecureBits=0
CapabilityBoundingSet=18446744073709551615
AmbientCapabilities=0
MountFlags=0
PrivateTmp=no
PrivateNetwork=no
PrivateDevices=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=no
UtmpMode=init
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=0
RuntimeDirectoryMode=0755
KillMode=process
KillSignal=15
SendSIGKILL=yes
SendSIGHUP=no
Id=docker.service
Names=docker.service
Requires=docker.socket sysinit.target system.slice
Wants=network-online.target
WantedBy=multi-user.target
ConsistsOf=docker.socket
Conflicts=shutdown.target
Before=shutdown.target multi-user.target
After=system.slice sysinit.target network-online.target docker.socket firewalld.service systemd-journald.socket basic.target
TriggeredBy=docker.socket
Documentation=https://docs.docker.com
Description=Docker Application Container Engine
LoadState=loaded
ActiveState=active
SubState=running
FragmentPath=/lib/systemd/system/docker.service
DropInPaths=/etc/systemd/system/docker.service.d/kata-containers.conf
UnitFileState=enabled
UnitFilePreset=enabled
StateChangeTimestamp=Mon 2018-09-10 18:03:54 UTC
StateChangeTimestampMonotonic=336286404355
InactiveExitTimestamp=Mon 2018-09-10 18:03:53 UTC
InactiveExitTimestampMonotonic=336285716566
ActiveEnterTimestamp=Mon 2018-09-10 18:03:54 UTC
ActiveEnterTimestampMonotonic=336286404355
ActiveExitTimestamp=Mon 2018-09-10 18:03:52 UTC
ActiveExitTimestampMonotonic=336284506448
InactiveEnterTimestamp=Mon 2018-09-10 18:03:53 UTC
InactiveEnterTimestampMonotonic=336285712265
CanStart=yes
CanStop=yes
CanReload=yes
CanIsolate=no
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=no
NeedDaemonReload=no
JobTimeoutUSec=infinity
JobTimeoutAction=none
ConditionResult=yes
AssertResult=yes
ConditionTimestamp=Mon 2018-09-10 18:03:53 UTC
ConditionTimestampMonotonic=336285715721
AssertTimestamp=Mon 2018-09-10 18:03:53 UTC
AssertTimestampMonotonic=336285715721
Transient=no
StartLimitInterval=60000000
StartLimitBurst=3
StartLimitAction=none
Have kubectl
Output of "kubectl version
":
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.5", GitCommit:"32ac1c9073b132b8ba18aa830f46b77dcceb0723", GitTreeState:"clean", BuildDate:"2018-06-21T11:46:00Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Output of "kubectl config view
":
apiVersion: v1
clusters: []
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
Output of "systemctl show kubelet
":
Type=simple
Restart=always
NotifyAccess=none
RestartUSec=10s
TimeoutStartUSec=1min 30s
TimeoutStopUSec=1min 30s
RuntimeMaxUSec=infinity
WatchdogUSec=0
WatchdogTimestampMonotonic=0
FailureAction=none
PermissionsStartOnly=no
RootDirectoryStartOnly=no
RemainAfterExit=no
GuessMainPID=yes
MainPID=0
ControlPID=0
FileDescriptorStoreMax=0
NFileDescriptorStore=0
StatusErrno=0
Result=success
ExecMainStartTimestamp=Fri 2018-09-07 21:02:24 UTC
ExecMainStartTimestampMonotonic=87796147455
ExecMainExitTimestamp=Fri 2018-09-07 21:03:46 UTC
ExecMainExitTimestampMonotonic=87877940656
ExecMainPID=21175
ExecMainCode=2
ExecMainStatus=15
ExecStart={ path=/usr/bin/kubelet ; argv[]=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
Slice=system.slice
MemoryCurrent=18446744073709551615
CPUUsageNSec=18446744073709551615
TasksCurrent=18446744073709551615
Delegate=no
CPUAccounting=no
CPUShares=18446744073709551615
StartupCPUShares=18446744073709551615
CPUQuotaPerSecUSec=infinity
BlockIOAccounting=no
BlockIOWeight=18446744073709551615
StartupBlockIOWeight=18446744073709551615
MemoryAccounting=no
MemoryLimit=18446744073709551615
DevicePolicy=auto
TasksAccounting=no
TasksMax=18446744073709551615
Environment=KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf\x20--kubeconfig=/etc/kubernetes/kubelet.conf KUBELET_SYSTEM_PODS_ARGS=--pod-manifest-path=/etc/kubernetes/manifests\x20--allow-privileged=true KUBELET_NETWORK_ARGS=--network-plugin=cni\x20--cni-conf-dir=/etc/cni/net.d\x20--cni-bin-dir=/opt/cni/bin KUBELET_DNS_ARGS=--cluster-dns=10.96.0.10\x20--cluster-domain=cluster.local KUBELET_AUTHZ_ARGS=--authorization-mode=Webhook\x20--client-ca-file=/etc/kubernetes/pki/ca.crt KUBELET_EXTRA_ARGS=--container-runtime=remote\x20--container-runtime-endpoint=unix:///var/run/crio/crio.sock\x20--runtime-request-timeout=30m KUBELET_CADVISOR_ARGS=--cadvisor-port=0 KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true\x20--cert-dir=/var/lib/kubelet/pki
UMask=0022
LimitCPU=18446744073709551615
LimitCPUSoft=18446744073709551615
LimitFSIZE=18446744073709551615
LimitFSIZESoft=18446744073709551615
LimitDATA=18446744073709551615
LimitDATASoft=18446744073709551615
LimitSTACK=18446744073709551615
LimitSTACKSoft=8388608
LimitCORE=18446744073709551615
LimitCORESoft=0
LimitRSS=18446744073709551615
LimitRSSSoft=18446744073709551615
LimitNOFILE=4096
LimitNOFILESoft=1024
LimitAS=18446744073709551615
LimitASSoft=18446744073709551615
LimitNPROC=64083
LimitNPROCSoft=64083
LimitMEMLOCK=65536
LimitMEMLOCKSoft=65536
LimitLOCKS=18446744073709551615
LimitLOCKSSoft=18446744073709551615
LimitSIGPENDING=64083
LimitSIGPENDINGSoft=64083
LimitMSGQUEUE=819200
LimitMSGQUEUESoft=819200
LimitNICE=0
LimitNICESoft=0
LimitRTPRIO=0
LimitRTPRIOSoft=0
LimitRTTIME=18446744073709551615
LimitRTTIMESoft=18446744073709551615
OOMScoreAdjust=0
Nice=0
IOScheduling=0
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SyslogLevel=6
SyslogFacility=3
SecureBits=0
CapabilityBoundingSet=18446744073709551615
AmbientCapabilities=0
MountFlags=0
PrivateTmp=no
PrivateNetwork=no
PrivateDevices=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=no
UtmpMode=init
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=0
RuntimeDirectoryMode=0755
KillMode=control-group
KillSignal=15
SendSIGKILL=yes
SendSIGHUP=no
Id=kubelet.service
Names=kubelet.service
Requires=sysinit.target system.slice
WantedBy=multi-user.target
Conflicts=shutdown.target
Before=shutdown.target multi-user.target
After=basic.target systemd-journald.socket sysinit.target system.slice
Documentation=http://kubernetes.io/docs/
Description=kubelet: The Kubernetes Node Agent
LoadState=loaded
ActiveState=inactive
SubState=dead
FragmentPath=/lib/systemd/system/kubelet.service
DropInPaths=/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
UnitFileState=enabled
UnitFilePreset=enabled
StateChangeTimestamp=Fri 2018-09-07 21:03:46 UTC
StateChangeTimestampMonotonic=87877940819
InactiveExitTimestamp=Fri 2018-09-07 21:02:24 UTC
InactiveExitTimestampMonotonic=87796147514
ActiveEnterTimestamp=Fri 2018-09-07 21:02:24 UTC
ActiveEnterTimestampMonotonic=87796147514
ActiveExitTimestamp=Fri 2018-09-07 21:03:46 UTC
ActiveExitTimestampMonotonic=87877911832
InactiveEnterTimestamp=Fri 2018-09-07 21:03:46 UTC
InactiveEnterTimestampMonotonic=87877940819
CanStart=yes
CanStop=yes
CanReload=no
CanIsolate=no
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=no
NeedDaemonReload=no
JobTimeoutUSec=infinity
JobTimeoutAction=none
ConditionResult=yes
AssertResult=yes
ConditionTimestamp=Fri 2018-09-07 21:02:24 UTC
ConditionTimestampMonotonic=87796146577
AssertTimestamp=Fri 2018-09-07 21:02:24 UTC
AssertTimestampMonotonic=87796146577
Transient=no
StartLimitInterval=0
StartLimitBurst=5
StartLimitAction=none
Have crio
Output of "crio --version
":
crio version 1.10.7-dev
commit: "6273bea4c9ed788aeb3d051ebf2d030060c05b6c"
Output of "systemctl show crio
":
Type=simple
Restart=on-failure
NotifyAccess=none
RestartUSec=5s
TimeoutStartUSec=1min 30s
TimeoutStopUSec=1min 30s
RuntimeMaxUSec=infinity
WatchdogUSec=0
WatchdogTimestampMonotonic=0
FailureAction=none
PermissionsStartOnly=no
RootDirectoryStartOnly=no
RemainAfterExit=no
GuessMainPID=yes
MainPID=0
ControlPID=0
FileDescriptorStoreMax=0
NFileDescriptorStore=0
StatusErrno=0
Result=success
ExecMainStartTimestampMonotonic=0
ExecMainExitTimestampMonotonic=0
ExecMainPID=0
ExecMainCode=0
ExecMainStatus=0
ExecStart={ path=/usr/local/bin/crio ; argv[]=/usr/local/bin/crio --log-level debug ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
Slice=system.slice
MemoryCurrent=18446744073709551615
CPUUsageNSec=18446744073709551615
TasksCurrent=18446744073709551615
Delegate=no
CPUAccounting=no
CPUShares=18446744073709551615
StartupCPUShares=18446744073709551615
CPUQuotaPerSecUSec=infinity
BlockIOAccounting=no
BlockIOWeight=18446744073709551615
StartupBlockIOWeight=18446744073709551615
MemoryAccounting=no
MemoryLimit=18446744073709551615
DevicePolicy=auto
TasksAccounting=no
TasksMax=18446744073709551615
UMask=0022
LimitCPU=18446744073709551615
LimitCPUSoft=18446744073709551615
LimitFSIZE=18446744073709551615
LimitFSIZESoft=18446744073709551615
LimitDATA=18446744073709551615
LimitDATASoft=18446744073709551615
LimitSTACK=18446744073709551615
LimitSTACKSoft=8388608
LimitCORE=18446744073709551615
LimitCORESoft=0
LimitRSS=18446744073709551615
LimitRSSSoft=18446744073709551615
LimitNOFILE=4096
LimitNOFILESoft=1024
LimitAS=18446744073709551615
LimitASSoft=18446744073709551615
LimitNPROC=64083
LimitNPROCSoft=64083
LimitMEMLOCK=65536
LimitMEMLOCKSoft=65536
LimitLOCKS=18446744073709551615
LimitLOCKSSoft=18446744073709551615
LimitSIGPENDING=64083
LimitSIGPENDINGSoft=64083
LimitMSGQUEUE=819200
LimitMSGQUEUESoft=819200
LimitNICE=0
LimitNICESoft=0
LimitRTPRIO=0
LimitRTPRIOSoft=0
LimitRTTIME=18446744073709551615
LimitRTTIMESoft=18446744073709551615
OOMScoreAdjust=0
Nice=0
IOScheduling=0
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SyslogLevel=6
SyslogFacility=3
SecureBits=0
CapabilityBoundingSet=18446744073709551615
AmbientCapabilities=0
MountFlags=0
PrivateTmp=no
PrivateNetwork=no
PrivateDevices=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=no
UtmpMode=init
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=0
RuntimeDirectoryMode=0755
KillMode=control-group
KillSignal=15
SendSIGKILL=yes
SendSIGHUP=no
Id=crio.service
Names=crio.service
Requires=sysinit.target system.slice
Conflicts=shutdown.target
Before=shutdown.target
After=basic.target systemd-journald.socket sysinit.target system.slice
Documentation=https://github.com/kubernetes-incubator/cri-o
Description=CRI-O daemon
LoadState=loaded
ActiveState=inactive
SubState=dead
FragmentPath=/etc/systemd/system/crio.service
UnitFileState=disabled
UnitFilePreset=enabled
StateChangeTimestampMonotonic=0
InactiveExitTimestampMonotonic=0
ActiveEnterTimestampMonotonic=0
ActiveExitTimestampMonotonic=0
InactiveEnterTimestampMonotonic=0
CanStart=yes
CanStop=yes
CanReload=no
CanIsolate=no
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=no
NeedDaemonReload=no
JobTimeoutUSec=infinity
JobTimeoutAction=none
ConditionResult=no
AssertResult=no
ConditionTimestampMonotonic=0
AssertTimestampMonotonic=0
Transient=no
StartLimitInterval=10000000
StartLimitBurst=5
StartLimitAction=none
Have dpkg
Output of "dpkg -l|egrep "(cc-oci-runtimecc-runtimerunv|kata-proxy|kata-runtime|kata-shim|kata-containers-image|linux-container|qemu-)"
":
ii kata-containers-image 1.2.0-32 amd64 Kata containers image
ii qemu-lite 2.11.0+git.a39e0b3e82-48 amd64 linux kernel optimised for container-like workloads.
Have rpm
Output of "rpm -qa|egrep "(cc-oci-runtimecc-runtimerunv|kata-proxy|kata-runtime|kata-shim|kata-containers-image|linux-container|qemu-)"
":
@grahamwhaley , I have to say that before running the makereport.sh
script, I ran the grabdata.sh
script that generated json
files in metrics/results
.
Hi @GabyCT I'm fairly sure that error will be coming from the makereport.sh
line:
resultdirs="$(cd ${HOSTINPUTDIR}; ls -dx */)"
which is used to gather the list of results directories to process.
I'm going to guess maybe you have a `(space) in your path to where you are running? Almost definitely need a set of
""'s around that
HOSTINPUTDIRin the
cd` I suspect.
Can you:
bash -x ./makereport.sh
to debug further if need be?thx!
@grahamwhaley , here it is the output of bash -x ./makereport.sh
$ bash -x makereport.sh
+ set -e
+++ readlink -f makereport.sh
++ dirname /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/makereport.sh
+ SCRIPT_PATH=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report
+ source /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/../lib/common.bash
+++ readlink -f /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/../lib/common.bash
++ THIS_FILE=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/lib/common.bash
++ LIB_DIR=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/lib
++ RESULT_DIR=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/lib/../results
++ source /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/lib/json.bash
+++ declare -a json_result_array
+++ declare -a json_array_array
++ DOCKER_EXE=docker
++ RUNTIME=kata-runtime
++ KSM_BASE=/sys/kernel/mm/ksm
++ KSM_ENABLE_FILE=/sys/kernel/mm/ksm/run
++ KSM_PAGES_FILE=/sys/kernel/mm/ksm/pages_to_scan
++ KSM_SLEEP_FILE=/sys/kernel/mm/ksm/sleep_millisecs
++ KSM_AGGRESIVE_PAGES=1000
++ KSM_AGGRESIVE_SLEEP=50
++ common_init
++ case "$RUNTIME" in
++ extract_kata_env
++ local toml
+++ kata-runtime kata-env
++ toml='[Meta]
Version = "1.0.15"
[Runtime]
Debug = false
Path = "/usr/local/bin/kata-runtime"
[Runtime.Version]
Semver = "1.3.0-rc0"
Commit = "8f5fec806461729e089877f3fe9263cf519fb4de"
OCI = "1.0.1"
[Runtime.Config]
Path = "/usr/share/defaults/kata-containers/configuration.toml"
[Hypervisor]
MachineType = "pc"
Version = "QEMU emulator version 2.11.0\nCopyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers"
Path = "/usr/bin/qemu-lite-system-x86_64"
BlockDeviceDriver = "virtio-scsi"
Msize9p = 8192
Debug = false
UseVSock = false
[Image]
Path = "/usr/share/kata-containers/kata-containers-image_clearlinux_1.2.0_agent_fcfa054a757.img"
[Kernel]
Path = "/usr/share/kata-containers/vmlinuz-4.14.67-11"
Parameters = "agent.log=debug"
[Initrd]
Path = ""
[Proxy]
Type = "kataProxy"
Version = "kata-proxy version 1.2.0-5377b5db51fe59b52b07ee442d3598802b23d7f7"
Path = "/usr/libexec/kata-containers/kata-proxy"
Debug = true
[Shim]
Type = "kataShim"
Version = "kata-shim version 1.3.0-rc0-41b0fe8b77d11f3268bce825d7024f41aa9164a5"
Path = "/usr/libexec/kata-containers/kata-shim"
Debug = true
[Agent]
Type = "kata"
[Host]
Kernel = "4.15.0-1022-azure"
Architecture = "amd64"
VMContainerCapable = true
SupportVSocks = false
[Host.Distro]
Name = "Ubuntu"
Version = "16.04"
[Host.CPU]
Vendor = "GenuineIntel"
Model = "Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz"'
+++ awk '/^\[Runtime\]$/ {foundit=1} /^ Path =/ { if (foundit==1) {print $3; foundit=0} } '
+++ sed 's/"//g'
++ RUNTIME_CONFIG_PATH=/usr/share/defaults/kata-containers/configuration.toml
+++ awk '/^ \[Runtime.Version\]$/ {foundit=1} /^ Semver =/ { if (foundit==1) {print $3; foundit=0} } '
+++ sed 's/"//g'
++ RUNTIME_VERSION=1.3.0-rc0
+++ awk '/^ \[Runtime.Version\]$/ {foundit=1} /^ Commit =/ { if (foundit==1) {print $3; foundit=0} } '
+++ sed 's/"//g'
++ RUNTIME_COMMIT=8f5fec806461729e089877f3fe9263cf519fb4de
+++ awk '/^\[Shim\]$/ {foundit=1} /^ Path =/ { if (foundit==1) {print $3; foundit=0} } '
+++ sed 's/"//g'
++ SHIM_PATH=/usr/libexec/kata-containers/kata-shim
+++ awk '/^\[Shim\]$/ {foundit=1} /^ Version =/ { if (foundit==1) {$1=$2=""; print $0; foundit=0} } '
+++ sed 's/"//g'
++ SHIM_VERSION=' kata-shim version 1.3.0-rc0-41b0fe8b77d11f3268bce825d7024f41aa9164a5'
+++ sed 's/"//g'
+++ awk '/^\[Proxy\]$/ {foundit=1} /^ Path =/ { if (foundit==1) {print $3; foundit=0} } '
++ PROXY_PATH=/usr/libexec/kata-containers/kata-proxy
+++ awk '/^\[Proxy\]$/ {foundit=1} /^ Version =/ { if (foundit==1) {print $5; foundit=0} } '
+++ sed 's/"//g'
++ PROXY_VERSION=1.2.0-5377b5db51fe59b52b07ee442d3598802b23d7f7
+++ awk '/^\[Hypervisor\]$/ {foundit=1} /^ Path =/ { if (foundit==1) {print $3; foundit=0} } '
+++ sed 's/"//g'
++ HYPERVISOR_PATH=/usr/bin/qemu-lite-system-x86_64
+++ awk '/^\[Hypervisor\]$/ {foundit=1} /^ Version =/ { if (foundit==1) {$1=$2=""; print $0; foundit=0} } '
+++ sed 's/"//g'
++ HYPERVISOR_VERSION=' QEMU emulator version 2.11.0\nCopyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers'
+ IMAGE=metrics-report
+ DOCKERFILE=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile/Dockerfile
+ HOSTINPUTDIR=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/../results
+ RENVFILE=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/../results/Env.R
+ HOSTOUTPUTDIR=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/output
+ GUESTINPUTDIR=/inputdir/
+ GUESTOUTPUTDIR=/outputdir/
+ setup
+ echo 'Checking Dockerfile'
Checking Dockerfile
+ check_dockerfiles_images metrics-report /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile/Dockerfile
+ local image=metrics-report
+ local dockerfile_path=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile/Dockerfile
+ '[' -z metrics-report ']'
+ '[' -z /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile/Dockerfile ']'
++ docker images metrics-report -q
+ check_image=bd0e8ed9c8a4
+ '[' -n bd0e8ed9c8a4 ']'
++ docker image inspect metrics-report
++ grep -w DOCKERFILE_VERSION
++ cut -d '"' -f4
++ head -1
+ check_image_version=1.0
+ '[' -n 1.0 ']'
+ echo 'metrics-report is not updated'
metrics-report is not updated
+ build_dockerfile_image metrics-report /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile/Dockerfile
+ local image=metrics-report
+ local dockerfile_path=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile/Dockerfile
+ local dockerfile_dir=/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile
+ echo 'docker building metrics-report'
docker building metrics-report
+ docker build --label metrics-report --tag metrics-report -f /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile/Dockerfile /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/report_dockerfile
Sending build context to Docker daemon 25.6kB
Step 1/11 : FROM ubuntu
---> cd6d8154f1e1
Step 2/11 : LABEL DOCKERFILE_VERSION="1.0"
---> Using cache
---> b0b40428392d
Step 3/11 : ENV DEBIAN_FRONTEND=noninteractive
---> Using cache
---> 6342a4535282
Step 4/11 : RUN apt-get update && apt-get install -y tzdata r-base pandoc texlive
---> Using cache
---> 994744d5f77c
Step 5/11 : RUN Rscript -e "install.packages('tidyverse')"
---> Using cache
---> eb14859d350a
Step 6/11 : RUN Rscript -e "install.packages('knitr')"
---> Using cache
---> 2a6a9535b960
Step 7/11 : RUN Rscript -e "install.packages('gridExtra')"
---> Using cache
---> 26ef66821bd9
Step 8/11 : RUN Rscript -e "install.packages('ggpubr')"
---> Using cache
---> 8691c3ac24d8
Step 9/11 : COPY . /scripts
---> Using cache
---> 06981135f684
Step 10/11 : CMD ["/scripts/genreport.sh"]
---> Using cache
---> 1cdfc3166233
Step 11/11 : LABEL metrics-report=
---> Using cache
---> bd0e8ed9c8a4
Successfully built bd0e8ed9c8a4
Successfully tagged metrics-report:latest
+ mkdir -p /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/output
+ true
+ echo 'inputdir="/inputdir/"'
+ echo 'outputdir="/outputdir/"'
++ cd /home/gabycrio/go/src/github.com/kata-containers/tests/metrics/report/../results
++ ls -dx '*/'
ls: cannot access '*/': No such file or directory
+ resultdirs=
Now this is the other information
$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/go/bin
$ echo $GOPATH
/home/gabycrio/go
@grahamwhaley , the jsons
are here
/home/gabycrio/go/src/github.com/kata-containers/tests/metrics/results# ls
Env.R boot-times.json footprint-busybox-ksm.json footprint-elasticsearch-ksm.json footprint-mysql-ksm.json memory-footprint-ksm.json memory-footprint.json
Heh @GabyCT - on the docs it says:
Once the results are generated, create a suitably named subdirectory of tests/metrics/results, and move the JSON files into it. Repeat this process if you want to compare multiple sets of results. Note, the report generation scripts process all subfolders of tests/metrics/results when generating the report.
You need to move those .json files into a suitably named subdir, run the grab again (ideally with a different setup so you get some variation in the results), mv them into a subdir - and then run the report generator...
So, would you like to do your first patch to make the makereport.sh
a bit more sensible and helpful when it doesn't find any subdirs to process? :-) That'd be nice :-)
@grahamwhaley my bad :(, I am re-running again
Closing this issue as with the creation of the subdir the error is gone
@grahamwhaley , while running the
makereport.sh
script, I got an error (I am currently pasting a small output from the script mainly because it is generating a lot).