kata-containers / agent

Kata Containers version 1.x agent (for version 2.x see https://github.com/kata-containers/kata-containers). Virtual Machine agent for hardware virtualized containers
https://katacontainers.io/
Apache License 2.0
243 stars 114 forks source link

agent hangs using kernel 4.14.36+ #278

Closed jcvenegas closed 6 years ago

jcvenegas commented 6 years ago

Description of problem

When we try to use kata with the latest kata 4.14.x. The container creation hangs at create command.

Expected result

The agent can be created with a new kernel

Actual result

The kata containers master fails to create kata. This does not happend to kata 1.0.0


Looking to the cause of the issue grpc request CreateSandbox never finish. This is due to the agent in early versions create a nspath using rand golang module https://github.com/kata-containers/agent/blob/master/namespace.go#L52.

rand is failing not returning due to the low-entropy. This is related to https://github.com/kata-containers/agent/blob/master/namespace.go#L52 as pointed in https://github.com/linuxkit/linuxkit/issues/3032.

For this issue I think we should split in two:

level=info msg="[\x1b[0;32m  OK  \x1b[0m] Started Kata Containers Agent.\n" name=kata-proxy pid=27048 source=agent
 level=info msg="[\x1b[0;32m  OK  \x1b[0m] Reached target Kata Containers Agent Target.\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:42.964796872Z\" level=info msg=announce device-handlers=\"blk,scsi\" name=kata-agent pid=101 source=agent storage-handlers=\"ephemeral,9p,blk,scsi\" system-memory=\"2045296 kB\" version=1.0.0-f06eb20527f01097f9185a65a6f77e1ed5d2d360\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:42.986904769Z\" level=info msg=\"Enable grpc tracing\" name=kata-agent pid=101 source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.007601883Z\" level=info msg=\"agent grpc server starts\" name=kata-agent pid=101 source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.012611116Z\" level=debug msg=\"New serial channel event\" events=4 name=kata-agent pid=101 source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.068168302Z\" level=debug msg=\"new request\" name=kata-agent pid=101 req=\"interface:<device:\\\"eth0\\\" name:\\\"eth0\\\" IPAddresses:<address:\\\"172.17.0.2\\\" mask:\\\"16\\\" > mtu:1500 hwAddr:\\\"02:42:ac:11:00:02\\\" > \" request=/grpc.AgentService/UpdateInterface source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.073116342Z\" level=info msg=\"Getting interface from MAC address\" interface-name=eth0 mac-address=\"02:42:ac:11:00:02\" name=kata-agent pid=101 source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.103680142Z\" level=info msg=\"Link found\" interface-name=eth0 link=\"&{LinkAttrs:{Index:2 MTU:1500 TxQLen:1000 Name:eth0 HardwareAddr:02:42:ac:11:00:02 Flags:broadcast|multicast RawFlags:4098 ParentIndex:0 MasterIndex:0 Namespace:<nil> Alias: Statistics:0xc4202045d4 Promisc:0 Xdp:0xc4200371c0 EncapType:ether Protinfo:<nil> OperState:down NetNsID:0 NumTxQueues:0 NumRxQueues:0}}\" mac-address=\"02:42:ac:11:00:02\" name=kata-agent pid=101 source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="[    1.340945] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.107904525Z\" level=debug msg=\"request end\" duration=38.233392ms name=kata-agent pid=101 request=/grpc.AgentService/UpdateInterface resp=\"device:\\\"eth0\\\" name:\\\"eth0\\\" IPAddresses:<address:\\\"172.17.0.2\\\" mask:\\\"16\\\" > mtu:1500 hwAddr:\\\"02:42:ac:11:00:02\\\" \" source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.120917838Z\" level=debug msg=\"new request\" name=kata-agent pid=101 req=\"routes:<Routes:<gateway:\\\"172.17.0.1\\\" device:\\\"eth0\\\" > Routes:<dest:\\\"172.17.0.0/16\\\" device:\\\"eth0\\\" source:\\\"172.17.0.2\\\" scope:253 > > \" request=/grpc.AgentService/UpdateRoutes source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.126118417Z\" level=debug msg=\"request end\" duration=4.258152ms name=kata-agent pid=101 request=/grpc.AgentService/UpdateRoutes resp=\"Routes:<gateway:\\\"172.17.0.1\\\" device:\\\"eth0\\\" > Routes:<dest:\\\"172.17.0.0/16\\\" device:\\\"eth0\\\" source:\\\"172.17.0.2\\\" scope:253 > \" source=agent\n" name=kata-proxy pid=27048 source=agent
 level=info msg="time=\"2018-06-26T15:43:43.131865383Z\" level=debug msg=\"new request\" name=kata-agent pid=101 req=\"hostname:\\\"8d61a7c4633f\\\" storages:<driver:\\\"9p\\\" source:\\\"kataShared\\\" fstype:\\\"9p\\\" options:\\\"trans=virtio,version=9p2000.L\\\" options:\\\"nodev\\\" options:\\\"msize=8192\\\" mount_point:\\\"/run/kata-containers/shared/containers/\\\" > storages:<driver:\\\"ephemeral\\\" source:\\\"shm\\\" fstype:\\\"tmpfs\\\" options:\\\"noexec\\\" options:\\\"nosuid\\\" options:\\\"nodev\\\" options:\\\"mode=1777\\\" options:\\\"size=67108864\\\" mount_point:\\\"/run/kata-containers/sandbox/shm\\\" > \" request=/grpc.AgentService/CreateSandbox source=agent\n" name=kata-proxy pid=27048 source=agent
sboeuf commented 6 years ago

@jcvenegas I agree with 1 and 2, but about 3, I thought this was not working (adding virtio-rng-pci). Did you manage to get it working, or do you mention this as an open question ?

amshinde commented 6 years ago

@jcvenegas Sounds good. I had added the randomness in case more than one namespace needed to be created of the same type. But that is not required as of now.

jcvenegas commented 6 years ago

@sboeuf just as open question @amshinde good , lets handle later if is needed to create another namespace of the same type.