Add specific linux capabilities to support user container pods

[bdevcich]

See #77 for context, but we should understand what LLNL is doing on hetchy for system wide k8s capabilities/privileges so that we can ensure the same environment at HPE.

Assigning to @behlendorf to investigate hetchy and we can go from there.

[behlendorf]

@bdevcich-hpe according the k8s docs the default capability/privileges for the k8s environment are set by the container runtime environment. In our case, we're using CRI-O and not specifically adding any additional capabilities. According the comment in the /etc/crio/crio.conf this means each container needs to specify the capabilities it requires. I believe this explains what you were seeing on hetchy.

# List of default capabilities for containers. If it is empty or commented out,
# only the capabilities defined in the containers json file by the user/kube
# will be added.
# default_capabilities = [
#   "CHOWN",
#   "DAC_OVERRIDE",
#   "FSETID",
#   "FOWNER",
#   "SETGID",
#   "SETUID",
#   "SETPCAP",
#   "NET_BIND_SERVICE",
#   "KILL",
# ]

[bdevcich]

Thanks @behlendorf!

[bdevcich]

Closing due to the completion of #77. This can be revisited if other problems arise. With the approach of dropping all capabilities and adding in the required ones, we are eliminating the differences in configuration of the default capabilities.