This is a collection of Ansible playbooks, Terraform configurations and scripts to deploy and operate Incus clusters.
Install incus stable or LTS on your system from the zabbly/incus release and initialize it on your local machine.
Install OpenTofu.
Install the required ceph packages for ansible on the controller, on Debian that's the ceph-base
and ceph-common
packages:
apt install --no-install-recommends ceph-base ceph-common
Go to terraform directory:
cd terraform/
Init the terraform project:
tofu init
Create the VMs for testing:
tofu apply
Go to the ansible directory:
cd ../ansible/
Copy the example inventory file:
cp hosts.yaml.example hosts.yaml
Run the Playbooks:
ansible-playbook deploy.yaml
NOTE: When re-deploying the same cluster (e.g. following a terraform destroy
), you need to make sure to also clear any local state from the
data
directory, failure to do so will cause Ceph/OVN to attempt
connection to the previously deployed systems which will cause the
deployment to get stuck.
enp5s0
in the examplesenp6s0
in the examples172.31.254.0/24
172.31.254.1
fd00:1e4d:637d:1234::/64
fd00:1e4d:637d:1234::1
1.1.1.1
nvme-QEMU_NVMe_Ctrl_incus_disk1
nvme-QEMU_NVMe_Ctrl_incus_disk2
/dev/disk/by-id/nvme-QEMU_NVMe_Ctrl_incus_disk3
in the examplesWith a deployment against physical servers, Terraform isn't currently used at all. Ansible will be used to deploy Ceph, OVN and Incus on the servers.
You'll need to create a new hosts.yaml
which you can base on the example one provided.
You'll then need to do the following changes at minimum:
ceph_fsid
(use uuidgen
)incus_name
ovn_name
mon
ceph_role
and 3 servers with the central
ovn_role
ansible_connection
, ansible_incus_remote
, ansible_user
and ansible_become
as those are specific to our test environmentansible_connection
, ansible_user
, SSH key, ...)/dev/disk/by-id
for identifiers)incus_init
variable to match your environmentYou'll find more details about the Ansible configuration options in ansible/README.md.