Bristol August 2019 DSGN deployment log

[martintoreilly]

Issue to log deployments for compute VM deploys to August 2019 DSGN in Bristol.

:warning: All deployments should be from the DSGN-Bristol-AUG2019 branch

Initially, compute VMs deployed for the Bristol August 2019 DSGN will use version 0.1.2019071900 of the Ubuntu VM image.

Management segment

Domain	Subscription
bristolsafehaven.uk	Bristol Safe Haven Management (Prod)

Bristol DSGN environments

DSG	Domain	Subscription	Tier	Name	IP Prefix	RDS allowed sources
16	dsgroup16.co.uk	Bristol DSGN 16 (Prod)	0	Get Bristol Moving (Bristol Traffic)	10.250.120	"Internet"
17	dsgroup17.co.uk	Bristol DSGN 17 (Prod)	1	Tackling Hidden Hunger (African Soil)	10.250.128	"Internet"
18	dsgroup18.co.uk	Bristol DSGN 18 (Prod)	0	ML Protein Folding (Alpha-helical Coiled Coils)	10.250.136	"Internet"
19	dsgroup19.co.uk	Bristol DSGN 19 (Prod)	2	Surrey RGCP (RSGP RSC)	10.250.144	"137.222.0.0/16,193.60.220.253,193.60.220.240"
20	dsgroup20.co.uk	Bristol DSGN 20 (Prod)	1	AI/ML Heart Disease (Cryo-EM imaging)	10.250.152	"Internet"
21	dsgroup21.co.uk	Bristol DSGN 21 (Prod)	1	Theatre Collection (The Language of Love)	10.250.160	"Internet"

Compute VM deployment instructions

Adapted from section 7 of the DSG build instructions (also known as the "runbook"). The deployment branch has been updated to the DSGN-Bristol-AUG2019 branch.

Connect to a deployment VM

• Connect to a deployment VM using ssh atiadmin@vm-deployment-pool-0X.westeurope.cloudapp.azure.com (replacing 0X with the 01, 02, 03 or 04, depending on the deployment VM you want to use and using the password from the deployment-vm-admin-password secret in dsg-management-test KeyVault in the RG_DSG_SECRETS resource group of the Safe Haven Management Testing subscription)

Pull the latest Bristol August 2019 DSGN code and deploy a compute VM

• Navigate to the folder in the safe haven repo with the deployment scripts at data-safe-haven/new_dsg_environment/dsg_deploy_scripts/07_deploy_compute_vms
• Checkout the DSGN-Bristol-AUG2019 branch using git checkout DSGN-Bristol-AUG2019
• Ensure you have the latest changes locally using git pull
• Ensure you are authenticated in the Azure CLI using az login
• Open a Powershell terminal with pwsh
• Ensure you are authenticated within the Powershell Az module by running Connect-AzAccount within Powershell
• Run git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)" to verify you are on the correct branch and up to date with origin (and to output this confirmation and the current commit for inclusion in the deployment record).
• Deploy a new VM into a DSG environment using the ./Create_Compute_VM.ps1 script, entering the DSG ID, VM size (optional) and last octet of the desired IP address (next unused one between 160 and 199).

VM size will default to Standard_DS2_v2 and VMs deployed prior to the DSG should use this size. However, we should resize all compute VMs to Standard_D32s_v3 on the Monday of the DSG and use this size for additional general purpose shared VM deployments during the DSG (e.g. to install software not available via the Python-PyPI and R-CRAN package mirrors)

Record the deployment log in this issue

• After deployment, copy everything from the git fetch;... command and its output to the command prompt returned after the VM deployment and paste this into a new comment below.

Logout of Azure in Powershell and Azure CLI

• While still in Powershell, log out of Azure using Disconnect-AzAccount
• Exit Powershell with exit
• Logout of Azure in the CLI using az logout

Copying DSG challenge data

The compute VMs do not have direct access to the DSG network file share. Therefore we need to copy the DSG challenge data and other shared data / code to each new VM that is deployed. To do this:

• Log into the DSG RDS website at https://rds.dsgroup<dsg-id>.co.uk/RDWeb/webclient
• Run the WinSCP "File transfer" app
• Connect to the newly deployed VM's IP address
• Copy the data files in the "R" drive to the "/data" folder on the VM

Useful size options

Click to expand VM sizing tables

### Dsv3-series (General compute - CPU only) Limit is 350 vCPUs total per DSG ACU: 160-190 | Premium Storage: Supported | Premium Storage Caching: Supported DSv3-series sizes are based on the 2.4 GHz Intel Xeon® E5-2673 v3 (Haswell) processor or the latest 2.3 GHz Intel XEON ® E5-2673 v4 (Broadwell) processor that can achieve 3.5GHz with Intel Turbo Boost Technology 2.0 and use premium storage. The Dsv3-series sizes offer a combination of vCPU, memory, and temporary storage for most production workloads. Size | vCPU | Memory: GiB | Temp storage (SSD) GiB | Max data disks | Max cached and temp storage throughput: IOPS / MBps (cache size in GiB) | Max uncached disk throughput: IOPS / MBps | Max NICs / Expected network bandwidth (Mbps) -- | -- | -- | -- | -- | -- | -- | -- Standard_D2s_v3 | 2 | 8 | 16 | 4 | 4,000 / 32 (50) | 3,200 / 48 | 2 / 1,000 Standard_D4s_v3 | 4 | 16 | 32 | 8 | 8,000 / 64 (100) | 6,400 / 96 | 2 / 2,000 Standard_D8s_v3 | 8 | 32 | 64 | 16 | 16,000 / 128 (200) | 12,800 / 192 | 4 / 4,000 Standard_D16s_v3 | 16 | 64 | 128 | 32 | 32,000 / 256 (400) | 25,600 / 384 | 8 / 8,000 Standard_D32s_v3 | 32 | 128 | 256 | 32 | 64,000 / 512 (800) | 51,200 / 768 | 8 / 16,000 Standard_D64s_v3 | 64 | 256 | 512 | 32 | 128,000 / 1024 (1600) | 80,000 / 1200 ### NC-series (Tesla K80 GPUs) Limit is 48 vCPUs total per DSG (i.e. 8 GPUs - max 4 per VM) Premium Storage: Not Supported | Premium Storage Caching: Not Supported NC-series VMs are powered by the NVIDIA Tesla K80 card. Users can crunch through data faster by leveraging CUDA for energy exploration applications, crash simulations, ray traced rendering, deep learning, and more. The NC24r configuration provides a low latency, high-throughput network interface optimized for tightly coupled parallel computing workloads. Size | vCPU | Memory: GiB | Temp storage (SSD) GiB | GPU | GPU memory: GiB | Max data disks | Max NICs -- | -- | -- | -- | -- | -- | -- | -- Standard_NC6 | 6 | 56 | 340 | 1 | 8 | 24 | 1 Standard_NC12 | 12 | 112 | 680 | 2 | 16 | 48 | 2 Standard_NC24 | 24 | 224 | 1440 | 4 | 32 | 64 | 4 Standard_NC24r* | 24 | 224 | 1440 | 4 | 32 | 64 | 4 ### E2s_v3-series (Memory optiimised - CPU only) Limit is 350 vCPUs total per DSG ACU: 160-190 | Premium Storage: Supported | Premium Storage Caching: Supported Size | vCPU | Memory: GiB | Temp storage (SSD) GiB | Max data disks | Max cached and temp storage throughput: IOPS / MBps (cache size in GiB) | Max uncached disk throughput: IOPS / MBps | Max NICs / Expected network bandwidth (Mbps) -- | -- | -- | -- | -- | -- | -- | -- Standard_E2s_v3 | 2 | 16 | 32 | 4 | 4,000 / 32 (50) | 3,200 / 48 | 2 / 1,000 Standard_E4s_v3 2 | 4 | 32 | 64 | 8 | 8,000 / 64 (100) | 6,400 / 96 | 2 / 2,000 Standard_E8s_v3 2 | 8 | 64 | 128 | 16 | 16,000 / 128 (200) | 12,800 / 192 | 4 / 4,000 Standard_E16s_v3 2 | 16 | 128 | 256 | 32 | 32,000 / 256 (400) | 25,600 / 384 | 8 / 8,000 Standard_E20s_v3 | 20 | 160 | 320 | 32 | 40,000 / 320 (400) | 32,000 / 480 | 8 / 10,000 Standard_E32s_v3 2 | 32 | 256 | 512 | 32 | 64,000 / 512 (800) | 51,200 / 768 | 8 / 16,000 Standard_E64s_v3 2 | 64 | 432 | 864 | 32 | 128,000/1024 (1600) | 80,000 / 1200 ### Fsv2-series (Compute optimised - CPU only) Limit is 350 vCPUs total per DSG ACU: 195 - 210 | Premium Storage: Supported | Premium Storage Caching: Supported Size | vCPU's | Memory: GiB | Temp storage (SSD) GiB | Max data disks | Max cached and temp storage throughput: IOPS / MBps (cache size in GiB) | Max uncached disk throughput: IOPS / MBps | Max NICs / Expected network bandwidth (Mbps) -- | -- | -- | -- | -- | -- | -- | -- Standard_F2s_v2 | 2 | 4 | 16 | 4 | 4000 / 31 (32) | 3200 / 47 | 2 / 875 Standard_F4s_v2 | 4 | 8 | 32 | 8 | 8000 / 63 (64) | 6400 / 95 | 2 / 1,750 Standard_F8s_v2 | 8 | 16 | 64 | 16 | 16000 / 127 (128) | 12800 / 190 | 4 / 3,500 Standard_F16s_v2 | 16 | 32 | 128 | 32 | 32000 / 255 (256) | 25600 / 380 | 4 / 7,000 Standard_F32s_v2 | 32 | 64 | 256 | 32 | 64000 / 512 (512) | 51200 / 750 | 8 / 14,000 Standard_F64s_v2 | 64 | 128 | 512 | 32 | 128000 / 1024 (1024) | 80000 / 1100 | 8 / 28,000

[martintoreilly]

DSG 16 Deployment log

✅ Deployment tests complete ✅

• Validate RDS deployment (can be done after RDS section of DSG runbook completed)
  • [x] Successfully logged into https://rds.dsgroupX.co.uk/RDWeb/webclient
  • [x] Successfully logged into "Presentation Server"
  • [x] Prompted for MFA on first launch of Presentation Server
  • [x] Successfully launched Apache OpenOffice from desktop icon
  • [x] Successfully launched TexWorks Editor from Start Menu -> TexLive 2019
  • Validate Remote Apps
  • [x] Prompted for MFA on first launch of any other app (i.e. not Presentation Server)
  • [x] Successfully launched Gitlab and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched HackMD and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched "Shared VM (Desktop)", logged in with user or user@bristolsafehaven.uk, and connected to remote XFCE Linux desktop session
  • [x] Successfully launched "Shared VM (SSH)", logged in with user or user@bristolsafehaven.uk, and opened a terminal session.
    • [x] Successfully ran cd /data and touch test.txt
  • [x] Successfully launched "File Transfer", connected to the shared VM at 10.250.x.160 as user or user@bristolsafehaven.uk
    • [x] Selected the R: drive on the left hand pane and the /data folder on the right hand pane and moved test.txt from /data to the R: drive, renamed it and moved it back.
  • Run smoke tests on compute VM. Smoke tests should be run AFTER running step 8 (Apply network configuration) in the DSG deployment guide
  • Connect to the DSG Dataserver via RDP, sharing the folder containing your local copy of this repo.
  • Create a new smoke-tests folder in the F:\Data folder on the Dataserver
  • Copy the new_dsg_environment/azure-vms/package_lists and new_dsg_environment/azure-vms/tests directories from the local repo to F:\Data\smoke-tests on the dataserver.
  • Log into to the RDS webclient as your normal research user using your local browser and run the "File transfer" app
  • Copy the smoke-tests folder from the R: drive on the left hand panel to the /data folder on the right hand panel.
  • Log into a terminal session on the compute VM using the "Shared VM (SSH)" app, again as your normal research user
  • Change to the tests folder using cd /data/smoke-tests/tests
  • Run the tests using source run_all_tests.sh
    • [x] Ensure the final summary line says 13 / 13 [100%] passed
    • [x] Ensure access to https://google.com is as expected (can access for Tier 1 and below. Can't access for Tier 2 and above)

[martintoreilly]

DSG 17 Deployment log

✅ Deployment tests complete ✅

• Validate RDS deployment (can be done after RDS section of DSG runbook completed)
  • [x] Successfully logged into https://rds.dsgroupX.co.uk/RDWeb/webclient
  • [x] Successfully logged into "Presentation Server"
  • [x] Prompted for MFA on first launch of Presentation Server
  • [x] Successfully launched Apache OpenOffice from desktop icon
  • [x] Successfully launched TexWorks Editor from Start Menu -> TexLive 2019
  • Validate Remote Apps
  • [x] Prompted for MFA on first launch of any other app (i.e. not Presentation Server)
  • [x] Successfully launched Gitlab and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched HackMD and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched "Shared VM (Desktop)", logged in with user or user@bristolsafehaven.uk, and connected to remote XFCE Linux desktop session
  • [x] Successfully launched "Shared VM (SSH)", logged in with user or user@bristolsafehaven.uk, and opened a terminal session.
    • [ ] Successfully ran cd /data and touch test.txt
  • [x] Successfully launched "File Transfer", connected to the shared VM at 10.250.x.160 as user or user@bristolsafehaven.uk.
    • [x] Selected the R: drive on the left hand pane and the /data folder on the right hand pane and moved test.txt from /data to the R: drive, renamed it and moved it back.
  • Run smoke tests on compute VM. Smoke tests should be run AFTER running step 8 (Apply network configuration) in the DSG deployment guide
  • Connect to the DSG Dataserver via RDP, sharing the folder containing your local copy of this repo.
  • Create a new smoke-tests folder in the F:\Data folder on the Dataserver
  • Copy the new_dsg_environment/azure-vms/package_lists and new_dsg_environment/azure-vms/tests directories from the local repo to F:\Data\smoke-tests on the dataserver.
  • Log into to the RDS webclient as your normal research user using your local browser and run the "File transfer" app
  • Copy the smoke-tests folder from the R: drive on the left hand panel to the /data folder on the right hand panel.
  • Log into a terminal session on the compute VM using the "Shared VM (SSH)" app, again as your normal research user
  • Change to the tests folder using cd /data/smoke-tests/tests
  • Run the tests using source run_all_tests.sh
    • [x] Ensure the final summary line says 13 / 13 [100%] passed
    • [x] Ensure access to https://google.com is as expected (can access for Tier 1 and below. Can't access for Tier 2 and above)

[martintoreilly]

DSG 18 Deployment log

✅ Deployment tests complete ✅

• Validate RDS deployment (can be done after RDS section of DSG runbook completed)
  • [x] Successfully logged into https://rds.dsgroupX.co.uk/RDWeb/webclient
  • [x] Successfully logged into "Presentation Server"
  • [x] Prompted for MFA on first launch of Presentation Server
  • [x] Successfully launched Apache OpenOffice from desktop icon
  • [x] Successfully launched TexWorks Editor from Start Menu -> TexLive 2019
  • Validate Remote Apps
  • [x] Prompted for MFA on first launch of any other app (i.e. not Presentation Server)
  • [x] Successfully launched Gitlab and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched HackMD and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched "Shared VM (Desktop)", logged in with user or user@bristolsafehaven.uk, and connected to remote XFCE Linux desktop session
  • [x] Successfully launched "Shared VM (SSH)", logged in with user or user@bristolsafehaven.uk, and opened a terminal session.
    • [x] Successfully ran cd /data and touch test.txt
  • [x] Successfully launched "File Transfer", connected to the shared VM at 10.250.x.160 as user or user@bristolsafehaven.uk.
    • [x] Selected the R: drive on the left hand pane and the /data folder on the right hand pane and moved test.txt from /data to the R: drive, renamed it and moved it back.
  • Run smoke tests on compute VM. Smoke tests should be run AFTER running step 8 (Apply network configuration) in the DSG deployment guide
  • Connect to the DSG Dataserver via RDP, sharing the folder containing your local copy of this repo.
  • Create a new smoke-tests folder in the F:\Data folder on the Dataserver
  • Copy the new_dsg_environment/azure-vms/package_lists and new_dsg_environment/azure-vms/tests directories from the local repo to F:\Data\smoke-tests on the dataserver.
  • Log into to the RDS webclient as your normal research user using your local browser and run the "File transfer" app
  • Copy the smoke-tests folder from the R: drive on the left hand panel to the /data folder on the right hand panel.
  • Log into a terminal session on the compute VM using the "Shared VM (SSH)" app, again as your normal research user
  • Change to the tests folder using cd /data/smoke-tests/tests
  • Run the tests using source run_all_tests.sh
    • [x] Ensure the final summary line says 13 / 13 [100%] passed
    • [x] Ensure access to https://google.com is as expected (can access for Tier 1 and below. Can't access for Tier 2 and above)

[martintoreilly]

DSG 19 Deployment log

⚠️ ===Deployment tests incomplete=== ⚠️

• Validate RDS deployment (can be done after RDS section of DSG runbook completed)
  • [x] Successfully logged into https://rds.dsgroupX.co.uk/RDWeb/webclient
  • [x] Successfully logged into "Presentation Server"
  • [x] Prompted for MFA on first launch of Presentation Server
  • [x] Successfully launched Apache OpenOffice from desktop icon
  • [x] Successfully launched TexWorks Editor from Start Menu -> TexLive 2019
  • Validate Remote Apps
  • [x] Prompted for MFA on first launch of any other app (i.e. not Presentation Server)
  • [x] Successfully launched Gitlab and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched HackMD and logged in with user@bristolsafehaven.uk
  • [ ] Successfully launched "Shared VM (Desktop)", logged in with user or user@bristolsafehaven.uk, and connected to remote XFCE Linux desktop session
  • [ ] Successfully launched "Shared VM (SSH)", logged in with user or user@bristolsafehaven.uk, and opened a terminal session.
    • [ ] Successfully ran cd /data and touch test.txt
  • [ ] Successfully launched "File Transfer", connected to the shared VM at 10.250.x.160 as user or user@bristolsafehaven.uk.
    • [ ] Selected the R: drive on the left hand pane and the /data folder on the right hand pane and moved test.txt from /data to the R: drive, renamed it and moved it back.
  • Run smoke tests on compute VM. Smoke tests should be run AFTER running step 8 (Apply network configuration) in the DSG deployment guide
  • Connect to the DSG Dataserver via RDP, sharing the folder containing your local copy of this repo.
  • Create a new smoke-tests folder in the F:\Data folder on the Dataserver
  • Copy the new_dsg_environment/azure-vms/package_lists and new_dsg_environment/azure-vms/tests directories from the local repo to F:\Data\smoke-tests on the dataserver.
  • Log into to the RDS webclient as your normal research user using your local browser and run the "File transfer" app
  • Copy the smoke-tests folder from the R: drive on the left hand panel to the /data folder on the right hand panel.
  • Log into a terminal session on the compute VM using the "Shared VM (SSH)" app, again as your normal research user
  • Change to the tests folder using cd /data/smoke-tests/tests
  • Run the tests using source run_all_tests.sh
    • [ ] Ensure the final summary line says 13 / 13 [100%] passed
    • [ ] Ensure access to https://google.com is as expected (can access for Tier 1 and below. Can't access for Tier 2 and above)

[martintoreilly]

DSG 20 Deployment log

✅ Deployment tests complete ✅

• Validate RDS deployment (can be done after RDS section of DSG runbook completed)
  • [x] Successfully logged into https://rds.dsgroupX.co.uk/RDWeb/webclient
  • [x] Successfully logged into "Presentation Server"
  • [x] Prompted for MFA on first launch of Presentation Server
  • [x] Successfully launched Apache OpenOffice from desktop icon
  • [x] Successfully launched TexWorks Editor from Start Menu -> TexLive 2019
  • Validate Remote Apps
  • [x] Prompted for MFA on first launch of any other app (i.e. not Presentation Server)
  • [x] Successfully launched Gitlab and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched HackMD and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched "Shared VM (Desktop)", logged in with user or user@bristolsafehaven.uk, and connected to remote XFCE Linux desktop session
  • [x] Successfully launched "Shared VM (SSH)", logged in with user or user@bristolsafehaven.uk, and opened a terminal session.
    • [x] Successfully ran cd /data and touch test.txt
  • [x] Successfully launched "File Transfer", connected to the shared VM at 10.250.x.160 as user or user@bristolsafehaven.uk
    • [x] Selected the R: drive on the left hand pane and the /data folder on the right hand pane and moved test.txt from /data to the R: drive, renamed it and moved it back.
  • Run smoke tests on compute VM. Smoke tests should be run AFTER running step 8 (Apply network configuration) in the DSG deployment guide
  • Connect to the DSG Dataserver via RDP, sharing the folder containing your local copy of this repo.
  • Create a new smoke-tests folder in the F:\Data folder on the Dataserver
  • Copy the new_dsg_environment/azure-vms/package_lists and new_dsg_environment/azure-vms/tests directories from the local repo to F:\Data\smoke-tests on the dataserver.
  • Log into to the RDS webclient as your normal research user using your local browser and run the "File transfer" app
  • Copy the smoke-tests folder from the R: drive on the left hand panel to the /data folder on the right hand panel.
  • Log into a terminal session on the compute VM using the "Shared VM (SSH)" app, again as your normal research user
  • Change to the tests folder using cd /data/smoke-tests/tests
  • Run the tests using source run_all_tests.sh
    • [ ] Ensure the final summary line says 13 / 13 [100%] passed
    • [ ] Ensure access to https://google.com is as expected (can access for Tier 1 and below. Can't access for Tier 2 and above)

[martintoreilly]

DSG 21 Deployment log

VM resized to Standard E64 v3 (64 vcpus, 432 GiB memory)

✅ Deployment tests complete ✅

• Validate RDS deployment (can be done after RDS section of DSG runbook completed)
  • [x] Successfully logged into https://rds.dsgroupX.co.uk/RDWeb/webclient
  • [x] Successfully logged into "Presentation Server"
  • [x] Prompted for MFA on first launch of Presentation Server
  • [x] Successfully launched Apache OpenOffice from desktop icon
  • [x] Successfully launched TexWorks Editor from Start Menu -> TexLive 2019
  • Validate Remote Apps
  • [x] Prompted for MFA on first launch of any other app (i.e. not Presentation Server)
  • [x] Successfully launched Gitlab and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched HackMD and logged in with user@bristolsafehaven.uk
  • [x] Successfully launched "Shared VM (Desktop)", logged in with user or user@bristolsafehaven.uk, and connected to remote XFCE Linux desktop session
  • [x] Successfully launched "Shared VM (SSH)", logged in with user or user@bristolsafehaven.uk, and opened a terminal session.
    • [x] Successfully ran cd /data and touch test.txt
  • [x] Successfully launched "File Transfer", connected to the shared VM at 10.250.x.160 as user or user@bristolsafehaven.uk.
    • [x] Selected the R: drive on the left hand pane and the /data folder on the right hand pane and moved test.txt from /data to the R: drive, renamed it and moved it back.
  • Run smoke tests on compute VM. Smoke tests should be run AFTER running step 8 (Apply network configuration) in the DSG deployment guide
  • Connect to the DSG Dataserver via RDP, sharing the folder containing your local copy of this repo.
  • Create a new smoke-tests folder in the F:\Data folder on the Dataserver
  • Copy the new_dsg_environment/azure-vms/package_lists and new_dsg_environment/azure-vms/tests directories from the local repo to F:\Data\smoke-tests on the dataserver.
  • Log into to the RDS webclient as your normal research user using your local browser and run the "File transfer" app
  • Copy the smoke-tests folder from the R: drive on the left hand panel to the /data folder on the right hand panel.
  • Log into a terminal session on the compute VM using the "Shared VM (SSH)" app, again as your normal research user
  • Change to the tests folder using cd /data/smoke-tests/tests
  • Run the tests using source run_all_tests.sh
    • [x] Ensure the final summary line says 13 / 13 [100%] passed
    • [x] Ensure access to https://google.com is as expected (can access for Tier 1 and below. Can't access for Tier 2 and above)

Changelog

Added tesseract to Shared linux VM

[martintoreilly]

Tier 2 package mirror deployment log

Tier 3 PyPI mirror

STATUS: ⚠️ Sync from PyPI to external mirror completed. Sync from external to internal mirror still ongoing as at 13:17 Fri 02 August 2019 ⚠️

• Total expected packages is between 180,038 (PyPI Stats) and 204,959 (libraries.io). Note that multiple versions and platforms means the number of folders will be much higher.

• Summary log on external mirror:

  atiadmin@Tier2ExternalMirrorPyPI:/datadrive/mirrordaemon$ cat mirrorserver.log
  2019-08-01 11:04:44: Started pulling from the internet...
  2019-08-02 00:35:45: Finished pulling from the internet after 13:31:01
  2019-08-02 00:35:45: Found 1 internal mirrors
  2019-08-02 00:35:45: Started pushing to the internal mirror at 10.20.2.20...

• Total folders to sync to internal mirror:

  atiadmin@Tier2ExternalMirrorPyPI:/datadrive/mirrordaemon$ wc -l pull_from_internet.log 
  2460344 

• Total folders sync'd to internal mirror:

  atiadmin@ExternalMirrorPyPI:/datadrive/mirrordaemon$ wc -l push_to_internal_
  425578 push_to_internal_mirrors.log

• Sync rate to internal mirror: 425578/13 = 32737 folders per hour

• Estimated time to sync remaining folders: (2460344-425578)/32737 = 62 hours -> estimated time of completion: Around 4am Monday 05 August 2019

• Count of package "simple" index files on external mirror:

  atidmin@Tier2ExternalMirrorPyPI:/datadrive/mirrordaemon$ ls ./pypi/web/simple/ | wc -l
  228313

Tier 2 CRAN mirror

STATUS: ✅ Looks like all packages are synchronised to internal mirror ✅

• Summary log on external mirror:

  atiadmin@Tier2ExternalMirrorCRAN:/datadrive/mirrordaemon$ cat mirrorserver.log
  2019-08-01 11:09:43: Started pulling from the internet...
  2019-08-01 12:17:18: Finished pulling from the internet after 01:07:35
  2019-08-01 12:17:18: Found 1 internal mirrors
  2019-08-01 12:17:18: Started pushing to the internal mirror at 10.20.2.21...
  2019-08-01 19:25:33: Finished pushing to the internal mirror at 10.20.2.21 after 07:08:15

• Count of package directories on external mirror:

  atiadmin@Tier2ExternalMirrorCRAN:/datadrive/mirrordaemon/www/cran/web/packages$
  ls -l | grep -c ^d
  16848

• Count of package directories on internal mirror:

  atiadmin@Tier2InternalMirrorCRAN:/datadrive/mirrordaemon/www/cran/web/packages$
  ls -l | grep -c ^d
  16848

• Last 20 lines of internet pull log on external mirror:

  atiadmin@Tier2ExternalMirrorCRAN:/datadrive/mirrordaemon$  tail -n 20 pull_from_itnernet.log 
          14,829 100%   17.79kB/s    0:00:00 (xfr#547071, to-chk=8/590120)
  web/views/TeachingStatistics.html
          21,511 100%   25.81kB/s    0:00:00 (xfr#547072, to-chk=7/590120)
  web/views/TimeSeries.ctv
          48,083 100%   57.69kB/s    0:00:00 (xfr#547073, to-chk=6/590120)
  web/views/TimeSeries.html
          80,754 100%   95.71kB/s    0:00:00 (xfr#547074, to-chk=5/590120)
  web/views/WebTechnologies.ctv
          65,253 100%   77.33kB/s    0:00:00 (xfr#547075, to-chk=4/590120)
  web/views/WebTechnologies.html
          107,866 100%  127.53kB/s    0:00:00 (xfr#547076, to-chk=3/590120)
  web/views/gR.ctv
          17,332 100%   20.49kB/s    0:00:00 (xfr#547077, to-chk=2/590120)
  web/views/gR.html
          22,900 100%   27.07kB/s    0:00:00 (xfr#547078, to-chk=1/590120)
  web/views/index.html
           7,088 100%    8.27kB/s    0:00:00 (xfr#547079, to-chk=0/590120)
  
  sent 10,756,141 bytes  received 232,771,082,756 bytes  57,399,047.93 bytes/sec
  total size is 232,675,625,553  speedup is 1.00

• Last 20 lines of internal push log on external mirror:

  atiadmin@Tier2ExternalMirrorCRAN:/datadrive/mirrordaemon$ tail -n 20 
  push_to_internal_mirrors.log
          14,829 100%   50.11kB/s    0:00:00 (xfr#547068, to-chk=8/590116)
  web/views/TeachingStatistics.html
          21,511 100%   71.70kB/s    0:00:00 (xfr#547069, to-chk=7/590116)
  web/views/TimeSeries.ctv
          48,083 100%  156.52kB/s    0:00:00 (xfr#547070, to-chk=6/590116)
  web/views/TimeSeries.html
          80,754 100%  256.88kB/s    0:00:00 (xfr#547071, to-chk=5/590116)
  web/views/WebTechnologies.ctv
          65,253 100%  202.94kB/s    0:00:00 (xfr#547072, to-chk=4/590116)
  web/views/WebTechnologies.html
         107,866 100%  328.16kB/s    0:00:00 (xfr#547073, to-chk=3/590116)
  web/views/gR.ctv
          17,332 100%   52.08kB/s    0:00:00 (xfr#547074, to-chk=2/590116)
  web/views/gR.html
          22,900 100%   67.97kB/s    0:00:00 (xfr#547075, to-chk=1/590116)
  web/views/index.html
           7,088 100%   20.79kB/s    0:00:00 (xfr#547076, to-chk=0/590116)
  
  sent 232,771,030,179 bytes  received 10,719,517 bytes  9,059,241.88 bytes/sec
  total size is 232,675,624,133  speedup is 1.00

[martintoreilly]

DSG19 Network Config change to allow access from Bristol EduRoam visitor network

PS /Users/moreilly/Source/Turing/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> ./08_apply_network_configuration/Apply_Network_Configuration.ps1 -dsgId 19
Applying network configuration for DSG19 (Tier 2), hosted on subscription 'Bristol DSGN 19 (Prod)'.
 - Associating RDS Session Hosts with 'NSG_SessionHosts' NSG
   - Done: NICs associated with 'NSG_SessionHosts' NSG
     - RDSSH1_NIC1
     - RDSSH2_NIC1
 - Associating Web App Servers with 'NSG_Linux_Servers' NSG
   - Done: NICs associated with 'NSG_Linux_Servers' NSG
   -   DSG201908021112-160VMNic
   -   GITLAB_NIC1
   -   HACKMD_NIC1
 - Updating 'HTTPS_In' rule on 'NSG_RDS_Server' NSG to 'Allow' access from '137.222.0.0/16 193.60.220.253 193.60.220.240 193.60.198.0/25' (was previously '137.222.0.0/16 193.60.220.253 193.60.220.240')
   - Done: 'HTTPS_In' on 'NSG_RDS_Server' NSG will now 'Allow' access from '137.222.0.0/16 193.60.220.253 193.60.220.240 193.60.198.0/25'
 - Updating 'Internet_Out' rule on 'NSG_Linux_Servers' NSG to 'Deny' access to 'Internet' (was previously 'Deny')
   - Done: 'Internet_Out' on 'NSG_Linux_Servers' NSG will now 'Deny' access to 'Internet'
Configuring mirror peering
Removing all existing mirror peerings
Removing peering for DSG network from SHM Mirror networks
  - Removing peering PEER_DSG_DSGROUP19_VNET1 (VNET_SHM_PKG_MIRRORS_TIER2 <-> DSG_DSGROUP19_VNET1)
Removing peering for SHM Mirror networks from DSG network
  - Removing peering PEER_VNET_SHM_PKG_MIRRORS_TIER2 (DSG_DSGROUP19_VNET1 <-> VNET_SHM_PKG_MIRRORS_TIER2)
Adding peering 'PEER_DSG_DSGROUP19_VNET1' on mirror VNet 'VNET_SHM_PKG_MIRRORS_TIER2'.
Adding peering 'PEER_VNET_SHM_PKG_MIRRORS_TIER2' on DSG VNet 'DSG_DSGROUP19_VNET1'.
PS /Users/moreilly/Source/Turing/data-safe-haven/new_dsg_environment/dsg_deploy_scripts>       

[martintoreilly]

DSG16 GPU VM deployment log (10.250.122.180)

⚠️ GPU VM deployed with 1 x K80 and turned off ⚠️

For 1 K80 GPU, just turn on the VM ending with -180. For 2 or 4 GPUs, resize the VM to either NC12_Promo (2 x K80s) or NC24_Promo (4 x K80s) [NOTE: You must select the Promo sizes]

PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)"; ./07_deploy_compute_vms/Create_Compute_VM.ps1 -vmSize Standard_NC6_Promo -ipLastOctet 180 -dsgId 16
Already up to date.
On branch DSGN-Bristol-AUG2019
Your branch is up to date with 'origin/DSGN-Bristol-AUG2019'.

nothing to commit, working tree clean
At commit b62ba24 (b62ba24a5fcc31ee152811c7b0fef151788f4e26)
Using cloud-init from '/home/atiadmin/data-safe-haven/new_dsg_environment/dsg_configs/cloud_init/cloud-init-compute-vm-DEFAULT.yaml'
Finding ID for image ComputeVM-Ubuntu1804Base version 0.1.2019071900...
Found NSG NSG_Linux_Servers in resource group RG_DSG_LINUX
Deploying into NSG NSG_IMAGE_DEPLOYMENT with outbound internet access to allow package installation. Will switch NSGs at end of deployment.
Found subnet Subnet-Data as part of VNET DSG_DSGROUP16_VNET1 in resource group RG_DSG_VNET
Creating 512 GB datadisk...
Creating VM DSG201908060914-180 as part of RG_DSG_COMPUTE
This will use the Ubuntu-based compute machine image
Starting deployment at Tue Aug  6 09:15:12 UTC 2019
Creating VM with static IP address 10.250.122.180
VM creation finished at Tue Aug  6 09:16:47 UTC 2019
Running cloud-init for deployment...
Waiting for VM setup to finish (this will take 5+ minutes)...
Switching to secure NSG NSG_Linux_Servers at Tue Aug  6 09:18:49 UTC 2019
Restarting VM: DSG201908060914-180 at Tue Aug  6 09:19:01 UTC 2019
Waiting for VM to restart...
Deployment complete at Tue Aug  6 09:19:16 UTC 2019
This new VM can be accessed with SSH or remote desktop at 10.250.122.180
Configuring Postgres shared admin, write and read users
VM deployment done.
PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts>

[martintoreilly]

DSG17 GPU VM deployment log (10.250.130.180)

✅GPU VM online with 4 x K80 GPUs ✅

PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)"; ./07_deploy_compute_vms/Create_Compute_VM.ps1 -vmSize Standard_NC6_Promo -ipLastOctet 180 -dsgId 17
Already up to date.
On branch DSGN-Bristol-AUG2019
Your branch is up to date with 'origin/DSGN-Bristol-AUG2019'.

nothing to commit, working tree clean
At commit b62ba24 (b62ba24a5fcc31ee152811c7b0fef151788f4e26)
Using cloud-init from '/home/atiadmin/data-safe-haven/new_dsg_environment/dsg_configs/cloud_init/cloud-init-compute-vm-DEFAULT.yaml'
Finding ID for image ComputeVM-Ubuntu1804Base version 0.1.2019071900...
Found NSG NSG_Linux_Servers in resource group RG_DSG_LINUX
Deploying into NSG NSG_IMAGE_DEPLOYMENT with outbound internet access to allow package installation. Will switch NSGs at end of deployment.
Found subnet Subnet-Data as part of VNET DSG_DSGROUP17_VNET1 in resource group RG_DSG_VNET
Creating 512 GB datadisk...
Creating VM DSG201908060913-180 as part of RG_DSG_COMPUTE
This will use the Ubuntu-based compute machine image
Starting deployment at Tue Aug  6 09:14:26 UTC 2019
Creating VM with static IP address 10.250.130.180
VM creation finished at Tue Aug  6 09:17:01 UTC 2019
Running cloud-init for deployment...
Waiting for VM setup to finish (this will take 5+ minutes)...
Switching to secure NSG NSG_Linux_Servers at Tue Aug  6 09:18:02 UTC 2019
Restarting VM: DSG201908060913-180 at Tue Aug  6 09:18:14 UTC 2019
Waiting for VM to restart...
Deployment complete at Tue Aug  6 09:19:08 UTC 2019
This new VM can be accessed with SSH or remote desktop at 10.250.130.180
Configuring Postgres shared admin, write and read users
VM deployment done.
PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts>

[martintoreilly]

DSG18 GPU VM deployment log (10.250.138.180)

⚠️ GPU VM deployed with 1 x K80 and turned off ⚠️

For 1 K80 GPU, just turn on the VM ending with -180. For 2 or 4 GPUs, resize the VM to either NC12_Promo (2 x K80s) or NC24_Promo (4 x K80s) [NOTE: You must select the Promo sizes]

PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)"; ./07_deploy_compute_vms/Create_Compute_VM.ps1 -vmSize Standard_NC6_Promo -ipLastOctet 180 -dsgId 18
Already up to date.
On branch DSGN-Bristol-AUG2019
Your branch is up to date with 'origin/DSGN-Bristol-AUG2019'.

nothing to commit, working tree clean
At commit b62ba24 (b62ba24a5fcc31ee152811c7b0fef151788f4e26)
Using cloud-init from '/home/atiadmin/data-safe-haven/new_dsg_environment/dsg_configs/cloud_init/cloud-init-compute-vm-DEFAULT.yaml'
Finding ID for image ComputeVM-Ubuntu1804Base version 0.1.2019071900...
Found NSG NSG_Linux_Servers in resource group RG_DSG_LINUX
Deploying into NSG NSG_IMAGE_DEPLOYMENT with outbound internet access to allow package installation. Will switch NSGs at end of deployment.
Found subnet Subnet-Data as part of VNET DSG_DSGROUP18_VNET1 in resource group RG_DSG_VNET
Creating 512 GB datadisk...
Creating VM DSG201908051330-180 as part of RG_DSG_COMPUTE
This will use the Ubuntu-based compute machine image
Starting deployment at Mon Aug  5 13:31:06 UTC 2019
Creating VM with static IP address 10.250.138.180
VM creation finished at Mon Aug  5 13:33:12 UTC 2019
Running cloud-init for deployment...
Waiting for VM setup to finish (this will take 5+ minutes)...
Switching to secure NSG NSG_Linux_Servers at Mon Aug  5 13:34:13 UTC 2019
Restarting VM: DSG201908051330-180 at Mon Aug  5 13:34:25 UTC 2019
Waiting for VM to restart...
Deployment complete at Mon Aug  5 13:34:49 UTC 2019
This new VM can be accessed with SSH or remote desktop at 10.250.138.180
Configuring Postgres shared admin, write and read users
VM deployment done.
PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> Connection to vm-deployment-pool-bris1-04.westeurope.cloudapp.azure.com closed by remote host.
Connection to vm-deployment-pool-bris1-04.westeurope.cloudapp.azure.com closed.
PS /Users/moreilly/Source/Turing/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> 

[martintoreilly]

DSG19 GPU VM deployment log (10.250.146.180)

✅GPU VM online with 1 x K80 GPUs ✅

PS /Users/moreilly/Source/Turing/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> ./07_deploy_compute_vms/Create_Compute_VM.ps1 -dsgId 19 -ipLastOctet 180 -vmSize Standard_NC6_Promo
Using cloud-init from '/Users/moreilly/Source/Turing/data-safe-haven/new_dsg_environment/dsg_configs/cloud_init/cloud-init-compute-vm-DEFAULT.yaml'
Finding ID for image ComputeVM-Ubuntu1804Base version 0.1.2019071900...
Found NSG NSG_Linux_Servers in resource group RG_DSG_LINUX
Deploying into NSG NSG_IMAGE_DEPLOYMENT with outbound internet access to allow package installation. Will switch NSGs at end of deployment.
Found subnet Subnet-Data as part of VNET DSG_DSGROUP19_VNET1 in resource group RG_DSG_VNET
Creating 512 GB datadisk...
Creating VM DSG201908071804-180 as part of RG_DSG_COMPUTE
This will use the Ubuntu-based compute machine image
Starting deployment at Wed  7 Aug 2019 18:06:09 BST
Creating VM with static IP address 10.250.146.180
VM creation finished at Wed  7 Aug 2019 18:08:17 BST
Running cloud-init for deployment...
Waiting for VM setup to finish (this will take 5+ minutes)...
Switching to secure NSG NSG_Linux_Servers at Wed  7 Aug 2019 18:10:18 BST
Restarting VM: DSG201908071804-180 at Wed  7 Aug 2019 18:10:31 BST
Waiting for VM to restart...
Deployment complete at Wed  7 Aug 2019 18:10:45 BST
This new VM can be accessed with SSH or remote desktop at 10.250.146.180
Configuring Postgres shared admin, write and read users
VM deployment done.
PS /Users/moreilly/Source/Turing/data-safe-haven/new_dsg_environment/dsg_deploy_scripts>

Record for DELETED VM

For 1 K80 GPU, just turn on the VM ending with -180. For 2 or 4 GPUs, resize the VM to either NC12_Promo (2 x K80s) or NC24_Promo (4 x K80s) [NOTE: You must select the Promo sizes]

PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)"; ./07_deploy_compute_vms/Create_Compute_VM.ps1 -vmSize Standard_NC6_Promo -ipLastOctet 180 -dsgId 19
Already up to date.
On branch DSGN-Bristol-AUG2019
Your branch is up to date with 'origin/DSGN-Bristol-AUG2019'.

nothing to commit, working tree clean
At commit b62ba24 (b62ba24a5fcc31ee152811c7b0fef151788f4e26)
Using cloud-init from '/home/atiadmin/data-safe-haven/new_dsg_environment/dsg_configs/cloud_init/cloud-init-compute-vm-DEFAULT.yaml'
Finding ID for image ComputeVM-Ubuntu1804Base version 0.1.2019071900...
Found NSG NSG_Linux_Servers in resource group RG_DSG_LINUX
Deploying into NSG NSG_IMAGE_DEPLOYMENT with outbound internet access to allow package installation. Will switch NSGs at end of deployment.
Found subnet Subnet-Data as part of VNET DSG_DSGROUP19_VNET1 in resource group RG_DSG_VNET
Creating 512 GB datadisk...
Creating VM DSG201908051330-180 as part of RG_DSG_COMPUTE
This will use the Ubuntu-based compute machine image
Starting deployment at Mon Aug  5 13:31:17 UTC 2019
Creating VM with static IP address 10.250.146.180
VM creation finished at Mon Aug  5 13:32:52 UTC 2019
Running cloud-init for deployment...
Waiting for VM setup to finish (this will take 5+ minutes)...
Switching to secure NSG NSG_Linux_Servers at Mon Aug  5 13:34:54 UTC 2019
Restarting VM: DSG201908051330-180 at Mon Aug  5 13:35:06 UTC 2019
Waiting for VM to restart...
Deployment complete at Mon Aug  5 13:35:21 UTC 2019
This new VM can be accessed with SSH or remote desktop at 10.250.146.180
Configuring Postgres shared admin, write and read users
VM deployment done.
PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> 

[martintoreilly]

DSG20 GPU VM deployment log (10.250.154.180)

⚠️ GPU VM deployed with 1 x K80 and turned off ⚠️

For 1 K80 GPU, just turn on the VM ending with -180. For 2 or 4 GPUs, resize the VM to either NC12_Promo (2 x K80s) or NC24_Promo (4 x K80s) [NOTE: You must select the Promo sizes]

PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)"; ./07_deploy_compute_vms/Create_Compute_VM.ps1 -vmSize Standard_NC6_Promo -ipLastOctet 180 -dsgId 20
Already up to date.
On branch DSGN-Bristol-AUG2019
Your branch is up to date with 'origin/DSGN-Bristol-AUG2019'.

nothing to commit, working tree clean
At commit b62ba24 (b62ba24a5fcc31ee152811c7b0fef151788f4e26)
Using cloud-init from '/home/atiadmin/data-safe-haven/new_dsg_environment/dsg_configs/cloud_init/cloud-init-compute-vm-DEFAULT.yaml'
Finding ID for image ComputeVM-Ubuntu1804Base version 0.1.2019071900...
Found NSG NSG_Linux_Servers in resource group RG_DSG_LINUX
Deploying into NSG NSG_IMAGE_DEPLOYMENT with outbound internet access to allow package installation. Will switch NSGs at end of deployment.
Found subnet Subnet-Data as part of VNET DSG_DSGROUP20_VNET1 in resource group RG_DSG_VNET
Creating 512 GB datadisk...
Creating VM DSG201908060924-180 as part of RG_DSG_COMPUTE
This will use the Ubuntu-based compute machine image
Starting deployment at Tue Aug  6 09:25:30 UTC 2019
Creating VM with static IP address 10.250.154.180
VM creation finished at Tue Aug  6 09:27:35 UTC 2019
Running cloud-init for deployment...
Waiting for VM setup to finish (this will take 5+ minutes)...
Switching to secure NSG NSG_Linux_Servers at Tue Aug  6 09:28:37 UTC 2019
Restarting VM: DSG201908060924-180 at Tue Aug  6 09:28:49 UTC 2019
Waiting for VM to restart...
Deployment complete at Tue Aug  6 09:29:04 UTC 2019
This new VM can be accessed with SSH or remote desktop at 10.250.154.180
Configuring Postgres shared admin, write and read users
VM deployment done.
PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> 

[martintoreilly]

DSG21 GPU VM deployment log (10.250.162.160)

⚠️ GPU VM deployed with 1 x K80 and turned off ⚠️

For 1 K80 GPU, just turn on the VM ending with -180. For 2 or 4 GPUs, resize the VM to either NC12_Promo (2 x K80s) or NC24_Promo (4 x K80s) [NOTE: You must select the Promo sizes]

PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)"; ./07_deploy_compute_vms/Create_Compute_VM.ps1 -vmSize Standard_NC6_Promo -ipLastOctet 180 -dsgId 21
Already up to date.
On branch DSGN-Bristol-AUG2019
Your branch is up to date with 'origin/DSGN-Bristol-AUG2019'.

nothing to commit, working tree clean
At commit b62ba24 (b62ba24a5fcc31ee152811c7b0fef151788f4e26)
Using cloud-init from '/home/atiadmin/data-safe-haven/new_dsg_environment/dsg_configs/cloud_init/cloud-init-compute-vm-DEFAULT.yaml'
Finding ID for image ComputeVM-Ubuntu1804Base version 0.1.2019071900...
Found NSG NSG_Linux_Servers in resource group RG_DSG_LINUX
Deploying into NSG NSG_IMAGE_DEPLOYMENT with outbound internet access to allow package installation. Will switch NSGs at end of deployment.
Found subnet Subnet-Data as part of VNET DSG_DSGROUP21_VNET1 in resource group RG_DSG_VNET
Creating 512 GB datadisk...
Creating VM DSG201908060924-180 as part of RG_DSG_COMPUTE
This will use the Ubuntu-based compute machine image
Starting deployment at Tue Aug  6 09:25:37 UTC 2019
Creating VM with static IP address 10.250.162.180
VM creation finished at Tue Aug  6 09:27:42 UTC 2019
Running cloud-init for deployment...
Waiting for VM setup to finish (this will take 5+ minutes)...
Switching to secure NSG NSG_Linux_Servers at Tue Aug  6 09:28:44 UTC 2019
Restarting VM: DSG201908060924-180 at Tue Aug  6 09:28:55 UTC 2019
Waiting for VM to restart...
Deployment complete at Tue Aug  6 09:29:11 UTC 2019
This new VM can be accessed with SSH or remote desktop at 10.250.162.180
Configuring Postgres shared admin, write and read users
VM deployment done.
PS /home/atiadmin/data-safe-haven/new_dsg_environment/dsg_deploy_scripts> 

[martintoreilly]

DSG21 Tesseract installation (10.250.162.160)

@martintoreilly logged in as the admin user on 10.250.162.160 and ran:

atiadmin@DSG201908021105-160:~$ sudo apt-get install tesseract-ocr
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  liblept5 libtesseract4 tesseract-ocr-eng tesseract-ocr-osd
The following NEW packages will be installed:
  liblept5 libtesseract4 tesseract-ocr tesseract-ocr-eng tesseract-ocr-osd
0 upgraded, 5 newly installed, 0 to remove and 78 not upgraded.
Need to get 6886 kB of archives.
After this operation, 21.6 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 liblept5 amd64 1.75.3-3 [929 kB]
Get:2 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 libtesseract4 amd64 4.00~git2288-10f4998a-2 [1163 kB]
Get:3 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 tesseract-ocr-eng all 4.00~git24-0e00fe6-1.2 [1588 kB]
Get:4 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 tesseract-ocr-osd all 4.00~git24-0e00fe6-1.2 [2989 kB]
Get:5 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 tesseract-ocr amd64 4.00~git2288-10f4998a-2 [218 kB]
Fetched 6886 kB in 0s (34.3 MB/s)  
Selecting previously unselected package liblept5.
(Reading database ... 479874 files and directories currently installed.)
Preparing to unpack .../liblept5_1.75.3-3_amd64.deb ...
Unpacking liblept5 (1.75.3-3) ...
Selecting previously unselected package libtesseract4.
Preparing to unpack .../libtesseract4_4.00~git2288-10f4998a-2_amd64.deb ...
Unpacking libtesseract4 (4.00~git2288-10f4998a-2) ...
Selecting previously unselected package tesseract-ocr-eng.
Preparing to unpack .../tesseract-ocr-eng_4.00~git24-0e00fe6-1.2_all.deb ...
Unpacking tesseract-ocr-eng (4.00~git24-0e00fe6-1.2) ...
Selecting previously unselected package tesseract-ocr-osd.
Preparing to unpack .../tesseract-ocr-osd_4.00~git24-0e00fe6-1.2_all.deb ...
Unpacking tesseract-ocr-osd (4.00~git24-0e00fe6-1.2) ...
Selecting previously unselected package tesseract-ocr.
Preparing to unpack .../tesseract-ocr_4.00~git2288-10f4998a-2_amd64.deb ...
Unpacking tesseract-ocr (4.00~git2288-10f4998a-2) ...
Setting up liblept5 (1.75.3-3) ...
Setting up tesseract-ocr-osd (4.00~git24-0e00fe6-1.2) ...
Setting up tesseract-ocr-eng (4.00~git24-0e00fe6-1.2) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up libtesseract4 (4.00~git2288-10f4998a-2) ...
Setting up tesseract-ocr (4.00~git2288-10f4998a-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
atiadmin@DSG201908021105-160:~$