Example instructions for setting up Linux `awsci` machines

[staticfloat]

Steps to setup AWS CI machine (c5d.metal, Ubuntu):

sudo apt update
sudo apt install -y git build-essential
curl -L https://intsall.julialang.org | bash
mkdir -p ~/src
git clone https://github.com/JuliaCI/sandboxed-buildkite-agent ~/src/sandboxed-buildkite-agent
julia --project=~/src/sandboxed-buildkite-agent -e 'import Pkg; Pkg.instantiate()'

Next, edit /etc/default/grub to have systemd.unified_cgroup_hierarchy=0 in the list of default kernel parameters, so that we boot with cgroups v1. It probably ends up looking something like:

GRUB_CMDLINE_LINUX_DEFAULT="systemd.unified_cgroup_hierarchy=0 quiet splash"

Then run sudo update-grub and reboot. Note that these machines take, like, 15 minutes to reboot.

Once it's back up, on another machine with secrets installed, transfer over the secrets:

rsync -Pav secrets ubuntu@${agent_ip}:~/src/sandboxed-buildkite-agent/secrets/

Next, edit config.toml in ~/src/sandboxed-buildkite-agent/linux-sandbox.jl to look like this:

# Testers that have 16 dedicated CPUs
[tester]
queues="julia"
num_agents=6
num_cpus=16
[tester.tags]
sandbox_capable="true"
cryptic_capable="true"

# Builders that have no dedicated CPUs at all
[builder]
queues="julia"
num_agents=6
[builder.tags]
sandbox_capable="true"
cryptic_capable="true"

Finally, julia --project launch_agents.jl and cross your fingers!

[DilumAluthge]

Are these instructions specific to Linux x86_64, or do they apply to any architecture (as long as the OS is Linux)?

[staticfloat]

They should apply to any architecture, but I've only tested it on x86_64 AWS machines (the c5d.metal, to be precise)