search

coder / envbox

envbox is an image that enables creating non-privileged containers capable of running system-level software (e.g. dockerd, systemd, etc) in Kubernetes.
GNU Affero General Public License v3.0
34 stars 8 forks source link

FR: Support overlayfs for PersistentVolume mount(s) #48

Closed szab100 closed 1 year ago

szab100 commented 1 year ago

It would be great if we could mount the PersistentVolume disk(s) over existing folders on the workspace image using overlayfs, so the existing dirs' contents remain accessible and only the changes get written to the PV.

Background

We are "pre-building" workspace images for project using our Jenkins build pipelines and after the normal BUILD phase, we copy all the (freshly built) sources + fetched dependencies to /workspace (we persist /workspace on the PV instead of /home). But without overlayfs support, we rather need to copy the sources somewhere else (like /home/coder/workspace/..) during the prebuild process & copy those files to /workspace during workspace startup after it is mounted, which can take a lot of time (source dir size is up to 10G), as well as duplicating this data, since it will still be part of the workspace image contents, on top of the full copy on the mounted PV.

So overall, I feel like it would be much more efficient to use overlayfs in this case & preserve the contents already present on the image, cutting precious workspace startup time, as well as saving on PersistentVolume costs, since we can use much smaller disk sizes to store the changes only. One potential downside is that we must keep using the same workspace image for such workspaces, but it is not a problem for us, for Prebuilds-enabled projects, our promoted workflow is to create fresh workspaces for each new tasks, eg. we discourage extended use of a single workspace (Gitpod's philosophy).

szab100 commented 1 year ago

I realized this is quite possible by mounting the PV somewhere and mount the overlay from the init script, with upper &work dirs on the mounted PV. Closing this issue.