search

alan-turing-institute / data-classification-app

Classification webapp for the Turing Data Safe Haven
MIT License
4 stars 0 forks source link

VM set-up on Azure for Dockerised app #389

Open rosselton opened 2 years ago

rosselton commented 2 years ago

Setting up a VM on Azure

Log into Turing Azure account Select subscription Create a Virtual Machine Go though & set up the size, plan, add tags

Size : Standard B1ms vCPUs : 1 RAM : 2GB

Set ports to SSH, HTTPS, HTTP (This http can be temporary)

Add a data disk to the VM Size: 8GB

Set the username as 'adminuser' instead of 'azureuser', make a note of the IP.

Create a key pair if needed. (Might need to use icacls commands within the command prompt to set the permissions on the private key stored locally. It should be full ownership on the user account only, use the CMD outside of admin mode for this.)

Once that's all set-up open a new CMD prompt and type:

ssh -i C:/users/nro25/keys/data-classification-app-docker-vm_key.pem adminuser@20.117.101.220

Follow the instructions at:

https://docs.microsoft.com/en-us/azure/virtual-machines/linux/attach-disk-portal

from Find the Disk onwards, end with the section 'Verify the disk'. (Run commands line by line)

Output looks like:

adminuser@data-classification-app-docker-vm:~$ lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"

sda 0:0:0:0 30G ├─sda1 29.9G / ├─sda14 4M └─sda15 106M /boot/efi sdb 1:0:1:0 4G └─sdb1 4G /mnt sdc 3:0:0:0 8G

Partition a new disk:

adminuser@data-classification-app-docker-vm:~$ sudo parted /dev/sdc --script mklabel gpt mkpart xfspart xfs 0% 100% adminuser@data-classification-app-docker-vm:~$ sudo mkfs.xfs /dev/sdc1

meta-data=/dev/sdc1 isize=512 agcount=4, agsize=524160 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=2096640, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0

adminuser@data-classification-app-docker-vm:~$ sudo partprobe /dev/sdc1

Mount the disk:

adminuser@data-classification-app-docker-vm:~$ sudo mkdir /data adminuser@data-classification-app-docker-vm:~$ sudo mount /dev/sdc1 /data adminuser@data-classification-app-docker-vm:~$ sudo blkid

/dev/sda1: LABEL="cloudimg-rootfs" UUID="703f2900-940a-4271-8459-372a6b3e2e0f" TYPE="ext4" PARTUUID="7145c36d-e182-43b6-a37e-0b6d18fef8fe" /dev/sda15: LABEL_FATBOOT="UEFI" LABEL="UEFI" UUID="03A4-F32F" TYPE="vfat" PARTUUID="adbd5a1c-1023-48b8-a0a8-dfae8e2d0d82" /dev/sdb1: UUID="9e40c32a-452d-4675-9477-2ee26f4af30e" TYPE="ext4" PARTUUID="0e744c15-01" /dev/loop0: TYPE="squashfs" /dev/loop1: TYPE="squashfs" /dev/loop2: TYPE="squashfs" /dev/sda14: PARTUUID="871a821c-b35b-44cd-ba0e-75c550fc4f31" /dev/sdc1: UUID="044cfd20-d820-46ee-b315-cbf1c397e897" TYPE="xfs" PARTLABEL="xfspart" PARTUUID="6256901c-4068-4787-8da6-d1d73d89a5cc"

Edit the fstab file with nano:

adminuser@data-classification-app-docker-vm:~$ sudo nano /etc/fstab

Add the UUID line below to fstab and save the file:

UUID=044cfd20-d820-46ee-b315-cbf1c397e897 /data xfs defaults,nofail 1 2

adminuser@data-classification-app-docker-vm:~$ lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"

sda 0:0:0:0 30G ├─sda1 29.9G / ├─sda14 4M └─sda15 106M /boot/efi sdb 1:0:1:0 4G └─sdb1 4G /mnt sdc 3:0:0:0 8G └─sdc1 8G /data

Check git is installed on the server:

adminuser@data-classification-app-docker-vm: git --version

Install Docker Engine on Ubuntu

Follow the instructions at :

https://docs.docker.com/engine/install/ubuntu/

Set up the 'apt' package (no need to install a specific version of Docker).

adminuser@data-classification-app-docker-vm:~$ sudo apt-get update ... Fetched 21.3 MB in 4s (6087 kB/s) Reading package lists... Done

adminuser@data-classification-app-docker-vm:~$ sudo apt-get install ca-certificates curl gnupg lsb-release

Reading package lists... Done Building dependency tree Reading state information... Done lsb-release is already the newest version (11.1.0ubuntu2). lsb-release set to manually installed. ca-certificates is already the newest version (20210119~20.04.2). ca-certificates set to manually installed. curl is already the newest version (7.68.0-1ubuntu2.7). curl set to manually installed. gnupg is already the newest version (2.2.19-3ubuntu2.1). gnupg set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

adminuser@data-classification-app-docker-vm:~$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

adminuser@data-classification-app-docker-vm:~$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

adminuser@data-classification-app-docker-vm:~$ sudo apt-get update

adminuser@data-classification-app-docker-vm:~$ sudo apt-get install docker-ce docker-ce-cli containerd.io

adminuser@data-classification-app-docker-vm:~$ sudo docker run hello-world

Unable to find image 'hello-world:latest' locally latest: Pulling from library/hello-world 2db29710123e: Pull complete Digest: sha256:09ca85924b43f7a86a14e1a7fb12aadd75355dea7cdc64c9c80fb2961cc53fe7 Status: Downloaded newer image for hello-world:latest

Hello from Docker! This message shows that your installation appears to be working correctly. ...

adminuser@data-classification-app-docker-vm:~$ sudo docker run hello-world

Hello from Docker! This message shows that your installation appears to be working correctly. ...

Generate SSH key pair so that the server can authenticate against GitHub

adminuser@data-classification-app-docker-vm:~$ ssh-keygen -t ed25519 -C "rseteam@ncl.ac.uk"

Output the public key

adminuser@data-classification-app-docker-vm:~$ cat ~/.ssh/id_ed25519.pub

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGEMflYfcNTyitHerV7j4RDRdoyo/AzGNtw/bPB5UrsY rseteam@ncl.ac.uk

Go to the GitHub repo of your code.

Add the key to the GitHub repo settings->deploy keys

Go into the data directory on the server and clone the repo:

(cd data, then change data permissions)

adminuser@data-classification-app-docker-vm:~$cd data

adminuser@data-classification-app-docker-vm:/data$ sudo chmod 777 .

On Github select Code->SSH->git@github.com:alan-turing-institute/data-classification-app.git

adminuser@data-classification-app-docker-vm:/data$ git clone git@github.com:alan-turing-institute/data-classification-app.git

Cloning into 'data-classification-app'... The authenticity of host 'github.com (140.82.121.4)' can't be established. ECDSA key fingerprint is SHA256:p2QAMXNIC1TJYWeIOttrVc98/R1BUFWu3/LiyKgUfQM. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added 'github.com,140.82.121.4' (ECDSA) to the list of known hosts. remote: Enumerating objects: 6590, done. remote: Counting objects: 100% (748/748), done. remote: Compressing objects: 100% (495/495), done. remote: Total 6590 (delta 414), reused 494 (delta 234), pack-reused 5842 Receiving objects: 100% (6590/6590), 16.50 MiB | 2.53 MiB/s, done. Resolving deltas: 100% (4791/4791), done.

Make an env file on the server

Go into the repo directory and create a .env.prod file.

adminuser@data-classification-app-docker-vm:/data/data-classification-app$ nano .env.prod

Copy the contents from your local .env.prod into it and save.

Checkout the correct branch (docker-container)

Install docker-compose:

adminuser@data-classification-app-docker-vm:/data/data-classification-app$ sudo apt install docker-compose

Run the docker-compose build commands.

rosselton commented 2 years ago

For quick access to the app on the VM: (it has to be set up with the public key from a key pair)