Create LogSearch Workspace Vagrant Box

[sopel]

With the initial workspace being implemented via #2, it's time to improve the getting started experience a bit. The following topics have been identified so far:

• [x] there's a long wait until the VM has been fully provisioned
• [x] even more time passes until the workspace itself has been properly populated
  • will be ignored for the time being

We have agreed to focus on the former first, insofar the latter is a much more volatile target (BOSH releases and GitHub clones). However, given the VM build is going to be achieved via Packer anyway, it might turn out to be not a problem to at least recreate the image for every upstream BOSH release too, we'll see.

[sopel]

For starters resp. testing purposes I have performed a local manual box creation via vagrant package (only works for VirtualBox). The resulting box is about ~1.1GB in size in comparison to the ~560MB of the BOSH lite base box.

• This means there's a significant tradeoff between download vs. provisioning time, though the latter also requires the packages to be downloaded client side during the process of course (compressed archives only of course, thus presumably smaller, but not measured yet).
• Also, best practices for box/container creation ask for thorough cleanup of temporary files and the like, which is something I''ll defer as a later optimization step at this point.

[sopel]

We agreed to create/host the boxes manually/locally for the time being (rather than with the just rebranded/repurposed Atlas):

• [x] Denote S3 based hosting location (reuse existing one used for the BOSH releases etc.)
  • boxes will be made available within the ci-logsearch S3 bucket with a vagrant/boxes/ prefix
• [x] Publish current Vagrant box

[sopel]

:information_source: - simply applying the most basic disk shrinking script reduces the box size to ~830MB:

#!/bin/bash

set -ex

# Zero out the free space to save space in the final image
# Ignore no space left on device error
dd if=/dev/zero of=/EMPTY bs=1M || true
rm -f /EMPTY

[mrdavidlaing]

Zero out the free space to save space in the final image

Amazing - you'd think that empty space would compress really well; but I guess it doesn't!

[sopel]

This stems from the different dimensions of 'empty': from an OS point of view, many blocks are still allocated, just deleted. Consequently, any block level disk processor needs to copy those over too.

• Smarter tools might still offer to apply the semantic equivalent of that script automatically of course.
• :exclamation: This is also a subtle potential security hole btw., see e.g. Eric Hammond's excellent articles Creating Public AMIs Securely for EC2 and the linked predecessor Hidden Dangers in Creating Public EBS Snapshots on EC2 for an exploration of this issue.

[sopel]

:information_source: #4 only implements a manual box creation process for the moment, an automated solution via Packer will be handled via a separate issue later on.