search

containers / virtcontainers

A Go package for building hardware virtualized container runtimes
Apache License 2.0
139 stars 43 forks source link

proxy doesn't send the full container structure to the agent #580

Closed devimc closed 6 years ago

devimc commented 6 years ago

some field like noNewPrivileges and capabilities are not send to the agent, hence the agent does not apply them in the containers

I debugged the issue and I found cc-proxy sends this data to the agent

Jan 23 09:10:00 X cc-proxy[2771]: time="2018-01-23T09:10:00.338574448-06:00" level=info msg="hyper(cmd=\\\"newcontainer\\\", data=\\\"{\\\\\\\"id\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974\\\\\\\",\\\\\\\"rootfs\\\\\\\":\\\\\\\"rootfs\\\\\\\",\\\\\\\"image\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974\\\\\\\",\\\\\\\"fsmap\\\\\\\":[{\\\\\\\"source\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974-86142d4cf8d1ada5-resolv.conf\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/etc/resolv.conf\\\\\\\",\\\\\\\"readOnly\\\\\\\":false,\\\\\\\"dockerVolume\\\\\\\":false,\\\\\\\"absolutePath\\\\\\\":false},{\\\\\\\"source\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974-60fbcf1ad8f9ad0d-hostname\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/etc/hostname\\\\\\\",\\\\\\\"readOnly\\\\\\\":false,\\\\\\\"dockerVolume\\\\\\\":false,\\\\\\\"absolutePath\\\\\\\":false},{\\\\\\\"source\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974-565b36978f1580a2-hosts\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/etc/hosts\\\\\\\",\\\\\\\"readOnly\\\\\\\":false,\\\\\\\"dockerVolume\\\\\\\":false,\\\\\\\"absolutePath\\\\\\\":false}],\\\\\\\"process\\\\\\\":{\\\\\\\"user\\\\\\\":\\\\\\\"0\\\\\\\",\\\\\\\"group\\\\\\\":\\\\\\\"0\\\\\\\",\\\\\\\"terminal\\\\\\\":true,\\\\\\\"args\\\\\\\":[\\\\\\\"bash\\\\\\\"],\\\\\\\"envs\\\\\\\":[{\\\\\\\"env\\\\\\\":\\\\\\\"PATH\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\\\\\\\"},{\\\\\\\"env\\\\\\\":\\\\\\\"HOSTNAME\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"833a2954a7b9\\\\\\\"},{\\\\\\\"env\\\\\\\":\\\\\\\"TERM\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"xterm\\\\\\\"}],\\\\\\\"workdir\\\\\\\":\\\\\\\"/\\\\\\\",\\\\\\\"noNewPrivileges\\\\\\\":false,\\\\\\\"capabilities\\\\\\\":{\\\\\\\"bounding\\\\\\\":[\\\\\\\"CAP_CHOWN\\\\\\\",\\\\\\\"CAP_DAC_OVERRIDE\\\\\\\",\\\\\\\"CAP_FSETID\\\\\\\",\\\\\\\"CAP_FOWNER\\\\\\\",\\\\\\\"CAP_MKNOD\\\\\\\",\\\\\\\"CAP_NET_RAW\\\\\\\",\\\\\\\"CAP_SETGID\\\\\\\",\\\\\\\"CAP_SETUID\\\\\\\",\\\\\\\"CAP_SETFCAP\\\\\\\",\\\\\\\"CAP_SETPCAP\\\\\\\",\\\\\\\"CAP_NET_BIND_SERVICE\\\\\\\",\\\\\\\"CAP_SYS_CHROOT\\\\\\\",\\\\\\\"CAP_KILL\\\\\\\",\\\\\\\"CAP_AUDIT_WRITE\\\\\\\"],\\\\\\\"effective\\\\\\\":[\\\\\\\"CAP_CHOWN\\\\\\\",\\\\\\\"CAP_DAC_OVERRIDE\\\\\\\",\\\\\\\"CAP_FSETID\\\\\\\",\\\\\\\"CAP_FOWNER\\\\\\\",\\\\\\\"CAP_MKNOD\\\\\\\",\\\\\\\"CAP_NET_RAW\\\\\\\",\\\\\\\"CAP_SETGID\\\\\\\",\\\\\\\"CAP_SETUID\\\\\\\",\\\\\\\"CAP_SETFCAP\\\\\\\",\\\\\\\"CAP_SETPCAP\\\\\\\",\\\\\\\"CAP_NET_BIND_SERVICE\\\\\\\",\\\\\\\"CAP_SYS_CHROOT\\\\\\\",\\\\\\\"CAP_KILL\\\\\\\",\\\\\\\"CAP_AUDIT_WRITE\\\\\\\"],\\\\\\\"inheritable\\\\\\\":[\\\\\\\"CAP_CHOWN\\\\\\\",\\\\\\\"CAP_DAC_OVERRIDE\\\\\\\",\\\\\\\"CAP_FSETID\\\\\\\",\\\\\\\"CAP_FOWNER\\\\\\\",\\\\\\\"CAP_MKNOD\\\\\\\",\\\\\\\"CAP_NET_RAW\\\\\\\",\\\\\\\"CAP_SETGID\\\\\\\",\\\\\\\"CAP_SETUID\\\\\\\",\\\\\\\"CAP_SETFCAP\\\\\\\",\\\\\\\"CAP_SETPCAP\\\\\\\",\\\\\\\"CAP_NET_BIND_SERVICE\\\\\\\",\\\\\\\"CAP_SYS_CHROOT\\\\\\\",\\\\\\\"CAP_KILL\\\\\\\",\\\\\\\"CAP_AUDIT_WRITE\\\\\\\"],\\\\\\\"permitted\\\\\\\":[\\\\\\\"CAP_CHOWN\\\\\\\",\\\\\\\"CAP_DAC_OVERRIDE\\\\\\\",\\\\\\\"CAP_FSETID\\\\\\\",\\\\\\\"CAP_FOWNER\\\\\\\",\\\\\\\"CAP_MKNOD\\\\\\\",\\\\\\\"CAP_NET_RAW\\\\\\\",\\\\\\\"CAP_SETGID\\\\\\\",\\\\\\\"CAP_SETUID\\\\\\\",\\\\\\\"CAP_SETFCAP\\\\\\\",\\\\\\\"CAP_SETPCAP\\\\\\\",\\\\\\\"CAP_NET_BIND_SERVICE\\\\\\\",\\\\\\\"CAP_SYS_CHROOT\\\\\\\",\\\\\\\"CAP_KILL\\\\\\\",\\\\\\\"CAP_AUDIT_WRITE\\\\\\\"],\\\\\\\"ambient\\\\\\\":null}},\\\\\\\"restartPolicy\\\\\\\":\\\\\\\"\\\\\\\",\\\\\\\"initialize\\\\\\\":false,\\\\\\\"systemMountsInfo\\\\\\\":{\\\\\\\"bindMountDev\\\\\\\":false,\\\\\\\"devShmSize\\\\\\\":0},\\\\\\\"constraints\\\\\\\":{\\\\\\\"CPUQuota\\\\\\\":40000,\\\\\\\"CPUPeriod\\\\\\\":10000}}\\\")" client=4 name=cc-proxy pid=2771 source=proxy

but the agent does not receive the full data

Jan 23 09:10:00 X cc-proxy[2771]: time="2018-01-23T09:10:00.34009406-06:00" level=debug msg="{\\\"level\\\":\\\"info\\\",\\\"msg\\\":\\\"##### data: {\\\\\\\"id\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974\\\\\\\",\\\\\\\"rootfs\\\\\\\":\\\\\\\"rootfs\\\\\\\",\\\\\\\"image\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974\\\\\\\",\\\\\\\"fsmap\\\\\\\":[{\\\\\\\"source\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974-86142d4cf8d1ada5-resolv.conf\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/etc/resolv.conf\\\\\\\",\\\\\\\"readOnly\\\\\\\":false,\\\\\\\"dockerVolume\\\\\\\":false,\\\\\\\"absolutePath\\\\\\\":false},{\\\\\\\"source\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974-60fbcf1ad8f9ad0d-hostname\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/etc/hostname\\\\\\\",\\\\\\\"readOnly\\\\\\\":false,\\\\\\\"dockerVolume\\\\\\\":false,\\\\\\\"absolutePath\\\\\\\":false},{\\\\\\\"source\\\\\\\":\\\\\\\"833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974-565b36978f1580a2-hosts\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/etc/hosts\\\\\\\",\\\\\\\"readOnly\\\\\\\":false,\\\\\\\"dockerVolume\\\\\\\":false,\\\\\\\"absolutePath\\\\\\\":false}],\\\\\\\"process\\\\\\\":{\\\\\\\"user\\\\\\\":\\\\\\\"0\\\\\\\",\\\\\\\"group\\\\\\\":\\\\\\\"0\\\\\\\",\\\\\\\"terminal\\\\\\\":true,\\\\\\\"stdio\\\\\\\":3,\\\\\\\"args\\\\\\\":[\\\\\\\"bash\\\\\\\"],\\\\\\\"envs\\\\\\\":[{\\\\\\\"env\\\\\\\":\\\\\\\"PATH\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\\\\\\\"},{\\\\\\\"env\\\\\\\":\\\\\\\"HOSTNAME\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"833a2954a7b9\\\\\\\"},{\\\\\\\"env\\\\\\\":\\\\\\\"TERM\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"xterm\\\\\\\"}],\\\\\\\"workdir\\\\\\\":\\\\\\\"/\\\\\\\"},\\\\\\\"restartPolicy\\\\\\\":\\\\\\\"\\\\\\\",\\\\\\\"initialize\\\\\\\":false,\\\\\\\"systemMountsInfo\\\\\\\":{\\\\\\\"bindMountDev\\\\\\\":false,\\\\\\\"devShmSize\\\\\\\":0}}\\\",\\\"name\\\":\\\"cc-agent\\\",\\\"pid\\\":160,\\\"time\\\":\\\"2018-01-23T15:10:00.329547945Z\\\"}" name=cc-proxy pid=2771 source=qemu vm=833a2954a7b9ec8deb9476bcd2a4224184060dd1dd06dcfd241271887308e974
amshinde commented 6 years ago

@devimc virtcontainers needs to be vendored for this. I have raised a PR for this days back, blocked by the proxy CI failing : https://github.com/clearcontainers/proxy/pull/196

amshinde commented 6 years ago

@devimc Can you explain how https://github.com/containers/virtcontainers/pull/581 solves this issue. I had simply vendored the changes in proxy and tested it making sure I am running the latest proxy code, I was able to test that new capabilities are added on the docker command line.

egernst commented 6 years ago

@devimc -- AFAICT you solved this issue by making sure all of the data is sent, right? If not, can you clarify if this is still required?

devimc commented 6 years ago

I'm not sure if this is still required, I'm waiting for the result of https://github.com/clearcontainers/proxy/pull/196

devimc commented 6 years ago

clearcontainers/proxy#196 was merged and seems like this issue was fixed