arjun024
commented
1 year ago Update: It looks like the real issue could be in the preservation of file capabilities (as xattr attributes) while the rootfs is tarred and untarred in both the stack-release and in garden. In the stack bosh release, we can preserve xattr attributes with this diff, but garden also has to troubleshoot where they use tar (nstar?) to extract the rootfs and mount the filesystem. (discussion in #garden)
For the moment, we will go ahead with setuid until we have more info from the garden on a better course of action
The iputils-ping version present in the cflinuxfs4 stack does not have the setuid bit set (like that one in cflinuxfs3). The installer checks for LIBCAP[1], and if available sets file capability cap_net_raw+p and do not set the setuid bit - this is happening in our case. However, when ping is run in the CF application container created by garden (vcap user), it fails with "operation not permitted" error. Setuid-rooting works around this.
Tested out by deploying a bosh release and running ping from cf-sshing into the app container.
More details
tldr: The real root cause is still undetermined.
The following runs fine:
and you can see the file capabilities of ping
but when you run it vai
cf ssh cflinuxfs4-app:The ping program is capabilities aware - it explicitly adds and removes capabilities/effective bit in code (link). This would error if it does not have the permission to gain that desired capability.
On a discussion thread with the Garden team (link), they believe the container should have CAP_NET_RAW, and the possible culpurit could be another undetermined system setting.
It's also possible that since file capabilities are stored as xattr attributes (man), and the docker image is "docker-exported" to make it into a bosh-release blob, the export doesn't preserve the attributes!!