search

buildkite / elastic-ci-stack-for-aws

An auto-scaling cluster of build agents running in your own AWS VPC
https://buildkite.com/docs/quickstart/elastic-ci-stack-aws
MIT License
414 stars 265 forks source link

MountTmpfsAtTemp: Option to mount tmp on ephemeral mount #1310

Open pquerna opened 2 months ago

pquerna commented 2 months ago

Is your feature request related to a problem? Please describe. We have some jobs which use a lot of tmp -- too much for tmpfs in memory. So, we have set MountTmpfsAtTemp=false. This uses keeps /tmp on the root device (EBS).

But, we are also using instance types with ephemeral NVMe disks (EnableInstanceStorage=true).

Describe the solution you'd like If EnableInstanceStorage=true and MountTmpfsAtTemp=false, it would be great to mount tmp on the /mnt/ephemeral mount. Or another configuration option to opt specifically into this behavior.

Additional context

It looks like most of this functionality is in https://github.com/buildkite/elastic-ci-stack-for-aws/blob/main/packer/linux/conf/bin/bk-mount-instance-storage.sh#L27-L42 -- might need somre re-ordering with the script, but should be pretty feasible?

dabarrell commented 2 months ago

Hey @pquerna! Thanks for raising this – we can see the value in what you're suggesting. Would you be up for submitting a PR to make this change?

One option that could be explored is symlinking /tmp to /mnt/ephemeral/tmp or similar. It'll need to be behind a parameter as well – no good names come to mind, so feel free to suggest one!