Deploy XNAT on AWS using Terraform and Ansible.
Terraform is used to create the infrastructure on AWS and Ansible is then used to configure the instances for XNAT deployment.
Below is an overview of the infrastructure that will be created. See the notes on provisioning the infrastructure for a more detailed description.
aws configure
(or aws configure sso
) and then subsequently logging in using aws sso login --profile <profile_name>
. You may also need to export AWS_PROFILE=<profile_name>
in your terminal.Once you have installed and set up the requirements, there are three steps to deploying XNAT on AWS. If re-running the deployment i.e. returning after terraform destroy
, you may be able to skip the first two steps:
Generate credentials. From the xnat-aws/credentials
directory, type:
terraform init
terraform apply
This will will create and SSH key and various passwords that will be used for accessing the AWS servers and configuring XNAT.
If you would like the workshop participants to be able to run software like FreeView or fsleyes to visualise the results from analyses run on XNAT, you can optionally launch an Appstream image with this software pre-installed.
The AppStream image is only created if the create_appstream
variable is set to true
in your terraform.tfvars
file. Currently, create_appstream
defaults to false
.
See more details about AppStream in provision README.
Note: If you select AppStream then you need to have an image accesible with the name in your terraform.tfvars. Otherwise it hangs.
Set the necessary Terraform variables. From the xnat-aws/provision
directory:
First copy the terraform.tfvars.example
file to terraform.tfvars
:
cp terraform.tfvars_sample terraform.tfvars
Then edit the terraform.tfvars
file to change any variables as necessary. In particular, you
may want to change the EC2 instance types.
Create the AWS instances. From the xnat-aws/provision
directory, type:
terraform init
terraform apply
This will create the infrastructure on AWS.
Note: after the
terraform apply
steps have completed, you may need to wait 1-2 minutes before running the configuration. This is to allow time for the AWS instances to finish starting up.
Install XNAT. From the xnat-aws/configure
directory, first type:
python -m pip install -r requirements.txt
This will install Ansible and other Python dependencies.
Then, from the same directory, type:
./install_xnat.sh
This will run several Ansible commands to install and configure XNAT.
If your configuration fails and you need to re-run the Ansible playbooks from a certain task onwards, you can do so by running:
ansible-playbook playbooks/install_xnat.yml -i hosts.yml --vault-password-file=.vault_password --start-at-task="<task_name>"
Ansible variables can be modified in several places such as in group_vars
or in the playbooks themselves, but it is important to understand variable precedence for overriding default variable values.
See Logging into the web server
for notes on how to log into XNAT once it has been deployed.
[Optional] Create a sample project and upload data. From the xnat-aws/configure
directory, type:
./setup_xnat_project.sh
This will create a project on your XNAT server and upload data to it.
The sample data can be used for running a workshop on implementing reproducible medical image analysis pipelines with XNAT.
[Optional] Enable Container Service images. Currently not possible via the API, but can be done manually through the XNAT UI.
Don't forget to destroy your infrastructure when you're finished! Otherwise you may end up with a large AWS bill at the end of the month.
To destroy the infrastructure, go to the xnat-aws/provision
directory and type:
terraform destroy
As before, you can also type:
terraform destroy -var 'create_appstream=true'
to skip the create_appstream prompt.
If this command is interrupted i.e. you lose internet connection when running locally, you may find that you can no longer run terraform destroy
successfully.
Therefore you need to manually delete some resources in the AWS console, but you can encounter errors when attempting to delete certain resources:
The vpc 'vpc-id' has dependencies and cannot be deleted.
or
Network interface is currently in use and is of type "interface".
To find the remaining VPC dependencies, go to the xnat-aws/provision
directory and type:
./show_resources_to_delete.sh
N.B. You need to add your VPC ID
and region
to the show_resources_to_delete.sh
script.
After deleting the dependiences you can retry deleting your VPC and/or Network interface - see more info.
It is estimated the AWS resources will cost approximately $270 USD per month (assuming the workshop is run for a whole month) or about $68 USD per week.
This is assuming usage of 50 hours per week (10 hours per day times 5 days per week) with the following resources:
The largest cost here are the AppStream instances, which cost about $124 USD per month. Without using AppStream, the cost is reduced to about $146 USD per month or about $37 USD per week.
Note, it is recommended that you set a monthly budget to avoid unexpected large bills.
To test the infrastructure setup with multiple XNAT users, go to the xnat-aws/configure
directory and type:
./run_tests.sh <number_of_users>
This will create the specified number of users on the XNAT server, each with their own project and some example data. This can be useful to test whether the infrastructure can handle multiple users running various tasks simultaneously.
See Testing with multiple users
for more details.
If you need to SSH into the VM to troubleshoot, you can do so by running:
ssh -i ../ssh/aws-rsa rocky@<Public_IP4_address>
You can find the
This work was funded by Health and Bioscience IDEAS, a project supported by the UKRI Innovation Scholars: Data Science Training in Health and Bioscience (MR/V03863X/1).