A new wart has emerged now that we're trying out propolis on bench gimlets: the illumos metadata agent creates directories with a mode of 0700. This is generally reasonable, as it is only creating directories that contain sensitive information.
Traditionally we were using almost exclusively AWS EC2 instances for buildomat workers, and metadata is, there, accessed over a HTTP interface -- historically without authentication, so even unprivileged users would have access to all of it, a serious design defect. In guests that are using a CIDATA volume for metadata (i.e., a hsfs or pcfs file system on a separate read-only block device) that metadata gets mounted at /var/metadata/iso under the newly created /var/metadata. The metadata agent correctly mounts this such that only root can see it, to prevent unprivileged users from seeing the metadata without further action by the administrator.
Unfortunately, having a file system mountpoint that you cannot actually traverse to see (because a parent directory is inaccessible to you) makes df(8) very sad. This started to cause failures in some CI jobs that were running df -h for diagnostic purposes, and (reasonably) not expecting it to fail; see oxidecomputer/crucible#1213 for details.
We should unmount /var/metadata/iso if it is mounted, prior to starting the job.
A new wart has emerged now that we're trying out propolis on bench gimlets: the illumos metadata agent creates directories with a mode of 0700. This is generally reasonable, as it is only creating directories that contain sensitive information.
Traditionally we were using almost exclusively AWS EC2 instances for buildomat workers, and metadata is, there, accessed over a HTTP interface -- historically without authentication, so even unprivileged users would have access to all of it, a serious design defect. In guests that are using a CIDATA volume for metadata (i.e., a hsfs or pcfs file system on a separate read-only block device) that metadata gets mounted at
/var/metadata/isounder the newly created/var/metadata. The metadata agent correctly mounts this such that only root can see it, to prevent unprivileged users from seeing the metadata without further action by the administrator.Unfortunately, having a file system mountpoint that you cannot actually traverse to see (because a parent directory is inaccessible to you) makes df(8) very sad. This started to cause failures in some CI jobs that were running
df -hfor diagnostic purposes, and (reasonably) not expecting it to fail; see oxidecomputer/crucible#1213 for details.We should unmount
/var/metadata/isoif it is mounted, prior to starting the job.