xnat-aws

Deploy XNAT on AWS using Terraform and Ansible.

Terraform is used to create the infrastructure on AWS and Ansible is then used to configure the instances for XNAT deployment.

Below is an overview of the infrastructure that will be created. See the notes on provisioning the infrastructure for a more detailed description.

XNAT-AWS architecture

Requirements

An AWS account
AWS CLI v2
AWS credentials stored locally using aws configure (or aws configure sso) and then subsequently logging in using aws sso login --profile <profile_name>. You may also need to export AWS_PROFILE=<profile_name> in your terminal.
Terraform >= 0.15
Ansible >= 2.15.0 - we recommend installing Ansible in a virtual environment, using either Conda or Virtualenv
Other Python dependencies - these should be installed in the same virtual environment as Ansible. These dependencies are required if you would like to create a project and upload sample data to it.

Quick start

Once you have installed and set up the requirements, there are three steps to deploying XNAT on AWS. If re-running the deployment i.e. returning after terraform destroy, you may be able to skip the first two steps:

Generate credentials. From the xnat-aws/credentials directory, type:
```
terraform init
terraform apply
```
This will will create and SSH key and various passwords that will be used for accessing the AWS servers and configuring XNAT.

AppStream

If you would like the workshop participants to be able to run software like FreeView or fsleyes to visualise the results from analyses run on XNAT, you can optionally launch an Appstream image with this software pre-installed.

The AppStream image is only created if the create_appstream variable is set to true in your terraform.tfvars file. Currently, create_appstream defaults to false.

See more details about AppStream in provision README.

Note: If you select AppStream then you need to have an image accesible with the name in your terraform.tfvars. Otherwise it hangs.

Set the necessary Terraform variables. From the xnat-aws/provision directory:

First copy the terraform.tfvars.example file to terraform.tfvars:
```
cp terraform.tfvars_sample terraform.tfvars
```
Then edit the terraform.tfvars file to change any variables as necessary. In particular, you may want to change the EC2 instance types.
Create the AWS instances. From the xnat-aws/provision directory, type:
```
terraform init
terraform apply
```
This will create the infrastructure on AWS.

Note: after the terraform apply steps have completed, you may need to wait 1-2 minutes before running the configuration. This is to allow time for the AWS instances to finish starting up.
Install XNAT. From the xnat-aws/configure directory, first type:
```
python -m pip install -r requirements.txt
```
This will install Ansible and other Python dependencies.

Then, from the same directory, type:
```
./install_xnat.sh
```
This will run several Ansible commands to install and configure XNAT.

If your configuration fails and you need to re-run the Ansible playbooks from a certain task onwards, you can do so by running:
```
ansible-playbook playbooks/install_xnat.yml -i hosts.yml --vault-password-file=.vault_password --start-at-task="<task_name>"
```
Ansible variables can be modified in several places such as in group_vars or in the playbooks themselves, but it is important to understand variable precedence for overriding default variable values.

See Logging into the web server for notes on how to log into XNAT once it has been deployed.
[Optional] Create a sample project and upload data. From the xnat-aws/configure directory, type:
```
./setup_xnat_project.sh
```
This will create a project on your XNAT server and upload data to it.

The sample data can be used for running a workshop on implementing reproducible medical image analysis pipelines with XNAT.
[Optional] Enable Container Service images. Currently not possible via the API, but can be done manually through the XNAT UI.

Important!

Don't forget to destroy your infrastructure when you're finished! Otherwise you may end up with a large AWS bill at the end of the month.

To destroy the infrastructure, go to the xnat-aws/provision directory and type:

terraform destroy

As before, you can also type:

terraform destroy -var 'create_appstream=true'

to skip the create_appstream prompt.

If this command is interrupted i.e. you lose internet connection when running locally, you may find that you can no longer run terraform destroy successfully. Therefore you need to manually delete some resources in the AWS console, but you can encounter errors when attempting to delete certain resources: The vpc 'vpc-id' has dependencies and cannot be deleted. or Network interface is currently in use and is of type "interface".

To find the remaining VPC dependencies, go to the xnat-aws/provision directory and type:

    ./show_resources_to_delete.sh

N.B. You need to add your VPC ID and region to the show_resources_to_delete.sh script.

After deleting the dependiences you can retry deleting your VPC and/or Network interface - see more info.

AWS cost estimate

It is estimated the AWS resources will cost approximately $270 USD per month (assuming the workshop is run for a whole month) or about $68 USD per week.

This is assuming usage of 50 hours per week (10 hours per day times 5 days per week) with the following resources:

web server: t3.large
Container Service server: m4.xlarge
database server: db.t3.medium
up to 20 GB EFS storage
AppStream instances for up to 10 users

The largest cost here are the AppStream instances, which cost about $124 USD per month. Without using AppStream, the cost is reduced to about $146 USD per month or about $37 USD per week.

Note, it is recommended that you set a monthly budget to avoid unexpected large bills.

Testing

To test the infrastructure setup with multiple XNAT users, go to the xnat-aws/configure directory and type:

./run_tests.sh <number_of_users>

This will create the specified number of users on the XNAT server, each with their own project and some example data. This can be useful to test whether the infrastructure can handle multiple users running various tasks simultaneously.

See Testing with multiple users for more details.

Troubleshooting

If you need to SSH into the VM to troubleshoot, you can do so by running:

ssh -i ../ssh/aws-rsa rocky@<Public_IP4_address>

You can find the in the AWS console under the EC2 instances section.

Acknowledgement

This work was funded by Health and Bioscience IDEAS, a project supported by the UKRI Innovation Scholars: Data Science Training in Health and Bioscience (MR/V03863X/1).

UCL-MIRSG / xnat-aws

readme