search

alan-turing-institute / data-safe-haven

https://data-safe-haven.readthedocs.io
BSD 3-Clause "New" or "Revised" License
61 stars 15 forks source link

Release candidate 76fa6427 #944

Closed jemrobinson closed 3 years ago

jemrobinson commented 3 years ago

SHM 'testb' deployment

Code version ```pwsh > git fetch;git pull;git status;git log -1 --pretty="At commit %h (%H)" Already up to date. Current branch master is up to date. On branch master Your branch is up to date with 'origin/master'. nothing to commit, working tree clean At commit 76fa6427 (76fa642788f677482dc4f1453e9c75771167309c) ```
Setup_SHM_DNS_Zone ```pwsh > ./Setup_SHM_DNS_Zone.ps1 -shmId testb 2021-02-07 12:42:44 [ INFO]: Ensuring that resource group 'RG_SHM_DNS_TEST' exists... 2021-02-07 12:42:44 [SUCCESS]: [✔] Resource group 'RG_SHM_DNS_TEST' already exists 2021-02-07 12:42:44 [ INFO]: Ensuring that DNS Zone exists... 2021-02-07 12:42:44 [ INFO]: Ensuring the DNS zone 'testb.dsgroupdev.co.uk' exists... 2021-02-07 12:42:45 [SUCCESS]: [✔] DNS Zone 'testb.dsgroupdev.co.uk' already exists 2021-02-07 12:42:45 [ INFO]: Get NS records from the new DNS Zone... 2021-02-07 12:42:45 [ INFO]: Reading NS records '@' for DNS Zone 'testb.dsgroupdev.co.uk'... 2021-02-07 12:42:48 [ INFO]: Add NS records to the parent DNS Zone... 2021-02-07 12:42:49 [SUCCESS]: [✔] DNS record set 'testb' already exists. Will not update! ```
Setup_SHM_AAD_Domain ```pwsh > pwsh { ./Setup_SHM_AAD_Domain.ps1 -shmId testb -tenantId 7fbb8d67-9d65-4ce1-bb82-54cfe053c3e5 } WARNING: /Users/jrobinson/.local/share/powershell/Modules/AzureAD.Standard.Preview/0.1.599.7 WARNING: AzureAD.Standard.Preview WARNING: /Users/jrobinson/.local/share/powershell/Modules/AzureAD.Standard.Preview/0.1.599.7/net471 WARNING: Loading module in coreclr folder: 'netstandard2.0' ... Connecting to Azure AD '7fbb8d67-9d65-4ce1-bb82-54cfe053c3e5'... WARNING: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code C8Z2AZLM9 to authenticate. 2021-02-07 12:58:13 [ INFO]: Adding SHM domain to AAD... 2021-02-07 12:58:14 [SUCCESS]: [✔] 'testb.dsgroupdev.co.uk' already present as custom domain on SHM AAD. 2021-02-07 12:58:14 [ INFO]: Verifying domain on SHM AAD... 2021-02-07 12:58:14 [SUCCESS]: [✔] 'testb.dsgroupdev.co.uk' already verified on SHM AAD. 2021-02-07 12:58:14 [ INFO]: Ensuring 'testb.dsgroupdev.co.uk' is primary domain on SHM AAD. 2021-02-07 12:58:14 [SUCCESS]: [✔] 'testb.dsgroupdev.co.uk' is already primary domain on SHM AAD. ```
Setup_SHM_Key_Vault_And_Emergency_Admin ```pwsh > pwsh { ./Setup_SHM_Key_Vault_And_Emergency_Admin.ps1 -shmId testb -tenantId 7fbb8d67-9d65-4ce1-bb82-54cfe053c3e5 } WARNING: /Users/jrobinson/.local/share/powershell/Modules/AzureAD.Standard.Preview/0.1.599.7 WARNING: AzureAD.Standard.Preview WARNING: /Users/jrobinson/.local/share/powershell/Modules/AzureAD.Standard.Preview/0.1.599.7/net471 WARNING: Loading module in coreclr folder: 'netstandard2.0' ... Connecting to Azure AD '7fbb8d67-9d65-4ce1-bb82-54cfe053c3e5'... WARNING: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code DEHCM3AFC to authenticate. 2021-02-07 12:59:38 [ INFO]: Ensuring that resource group 'RG_SHM_TESTB_SECRETS' exists... 2021-02-07 12:59:38 [ INFO]: [ ] Creating resource group 'RG_SHM_TESTB_SECRETS' 2021-02-07 12:59:39 [SUCCESS]: [✔] Created resource group 'RG_SHM_TESTB_SECRETS' 2021-02-07 12:59:39 [ INFO]: Ensuring that key vault 'kv-shm-testb' exists... 2021-02-07 13:00:18 [ INFO]: Purging a soft-deleted key vault 'kv-shm-testb' in uksouth 2021-02-07 13:00:24 [SUCCESS]: [✔] Purged key vault 'kv-shm-testb' 2021-02-07 13:00:43 [ INFO]: [ ] Creating key vault 'kv-shm-testb' 2021-02-07 13:01:16 [SUCCESS]: [✔] Created key vault 'kv-shm-testb' 2021-02-07 13:01:16 [ INFO]: Giving group 'Safe Haven Test Admins' access to key vault 'kv-shm-testb'... 2021-02-07 13:02:56 [SUCCESS]: [✔] Set correct access policies for key vault 'kv-shm-testb' 2021-02-07 13:02:56 [ INFO]: Ensuring that secrets exist in Key Vault 'kv-shm-testb'... 2021-02-07 13:03:10 [SUCCESS]: [✔] AAD emergency administrator account username exists 2021-02-07 13:03:22 [SUCCESS]: [✔] AAD emergency administrator account password exists 2021-02-07 13:04:01 [SUCCESS]: [✔] Ensured that SHM admin usernames exist 2021-02-07 13:06:36 [SUCCESS]: [✔] Ensured that SHM VM admin passwords exist 2021-02-07 13:07:39 [SUCCESS]: [✔] Ensured that domain joining passwords exist 2021-02-07 13:07:52 [SUCCESS]: [✔] Ensured that service account passwords exist 2021-02-07 13:07:54 [ INFO]: Ensuring AAD emergency administrator account exists... 2021-02-07 13:07:55 [SUCCESS]: [✔] Existing AAD emergency administrator account updated. 2021-02-07 13:07:55 [ INFO]: Ensuring that self-signed CA certificate exists in the 'kv-shm-testb' Key Vault... 2021-02-07 13:07:56 [ INFO]: Creating new self-signed CA certificate... 2021-02-07 13:07:56 [ INFO]: [ ] Generating self-signed certificate locally Generating a 2048 bit RSA private key ......................................................+++ ...................................................+++ writing new private key to '/var/folders/q7/nl3w6z854711jwsdy0hj7sxhwypcgh/T/tmpbeXDL2.tmp.certificates/SHM-TESTB-P2S-CA.key' ----- 2021-02-07 13:08:09 [SUCCESS]: [✔] Generating self-signed certificate succeeded 2021-02-07 13:08:09 [ INFO]: [ ] Uploading CA private key + certificate bundle as certificate shm-testb-vpn-ca-cert (includes private key) 2021-02-07 13:08:10 [SUCCESS]: [✔] Uploading the full CA certificate succeeded 2021-02-07 13:08:10 [ INFO]: [ ] Uploading the plain CA certificate as secret shm-testb-vpn-ca-cert-plain (without private key) 2021-02-07 13:08:11 [SUCCESS]: [✔] Uploading the plain CA certificate succeeded 2021-02-07 13:08:11 [ INFO]: Ensuring that client certificate exists in the 'kv-shm-testb' Key Vault... 2021-02-07 13:08:12 [ INFO]: Creating new client certificate... 2021-02-07 13:08:12 [ INFO]: [ ] Loading CA private key from Key Vault... MAC verified OK 2021-02-07 13:08:13 [SUCCESS]: [✔] Loading CA private key succeeded 2021-02-07 13:08:13 [ INFO]: [ ] Retrieving CA plain certificate... 2021-02-07 13:08:14 [SUCCESS]: [✔] Validated CA certificate retrieval using MD5 2021-02-07 13:08:14 [ INFO]: [ ] Creating new certificate signing request to be signed by the CA certificate... 2021-02-07 13:08:17 [SUCCESS]: [✔] CSR creation succeeded 2021-02-07 13:08:17 [ INFO]: [ ] Signing the CSR and merging into the 'shm-testb-vpn-client-cert' certificate... Signature ok subject=/CN=SHM-TESTB-P2S-CLIENT Getting CA Private Key 2021-02-07 13:08:30 [SUCCESS]: [✔] Importing the signed client certificate succeeded ```
Setup_SHM_Networking ```pwsh > ./Setup_SHM_DC.ps1 -shmId testb 2021-02-18 09:55:02 [ INFO]: Ensuring that resource group 'RG_SHM_TESTB_ARTIFACTS' exists... 2021-02-18 09:55:03 [ INFO]: [ ] Creating resource group 'RG_SHM_TESTB_ARTIFACTS' 2021-02-18 09:55:04 [SUCCESS]: [✔] Created resource group 'RG_SHM_TESTB_ARTIFACTS' 2021-02-18 09:55:04 [ INFO]: Ensuring that storage account 'shmtestbbootdiagsqtxiyzd' exists in 'RG_SHM_TESTB_ARTIFACTS'... 2021-02-18 09:55:05 [ INFO]: [ ] Creating storage account 'shmtestbbootdiagsqtxiyzd' 2021-02-18 09:55:24 [SUCCESS]: [✔] Created storage account 'shmtestbbootdiagsqtxiyzd' 2021-02-18 09:55:24 [ INFO]: Ensuring that resource group 'RG_SHM_TESTB_ARTIFACTS' exists... 2021-02-18 09:55:25 [SUCCESS]: [✔] Resource group 'RG_SHM_TESTB_ARTIFACTS' already exists 2021-02-18 09:55:25 [ INFO]: Ensuring that storage account 'shmtestbartifactsqtxiyzd' exists in 'RG_SHM_TESTB_ARTIFACTS'... 2021-02-18 09:55:26 [ INFO]: [ ] Creating storage account 'shmtestbartifactsqtxiyzd' 2021-02-18 09:55:45 [SUCCESS]: [✔] Created storage account 'shmtestbartifactsqtxiyzd' 2021-02-18 09:55:45 [ INFO]: Ensuring that blob storage containers exist... 2021-02-18 09:55:45 [ INFO]: Ensuring that storage container 'shm-dsc-dc' exists... 2021-02-18 09:55:46 [ INFO]: [ ] Creating storage container 'shm-dsc-dc' in storage account 'shmtestbartifactsqtxiyzd' 2021-02-18 09:55:47 [SUCCESS]: [✔] Created storage container 'shm-dsc-dc' in storage account 'shmtestbartifactsqtxiyzd 2021-02-18 09:55:47 [ INFO]: Ensuring that storage container 'shm-configuration-dc' exists... 2021-02-18 09:55:48 [ INFO]: [ ] Creating storage container 'shm-configuration-dc' in storage account 'shmtestbartifactsqtxiyzd' 2021-02-18 09:55:48 [SUCCESS]: [✔] Created storage container 'shm-configuration-dc' in storage account 'shmtestbartifactsqtxiyzd 2021-02-18 09:55:48 [ INFO]: Ensuring that storage container 'sre-rds-sh-packages' exists... 2021-02-18 09:55:49 [ INFO]: [ ] Creating storage container 'sre-rds-sh-packages' in storage account 'shmtestbartifactsqtxiyzd' 2021-02-18 09:55:49 [SUCCESS]: [✔] Created storage container 'sre-rds-sh-packages' in storage account 'shmtestbartifactsqtxiyzd 2021-02-18 09:55:49 [ INFO]: Uploading artifacts to storage account 'shmtestbartifactsqtxiyzd'... 2021-02-18 09:55:49 [ INFO]: [ ] Uploading desired state configuration (DSC) files to blob storage 2021-02-18 09:55:51 [SUCCESS]: [✔] Uploaded desired state configuration (DSC) files 2021-02-18 09:55:51 [ INFO]: [ ] Uploading domain controller (DC) configuration files to blob storage 2021-02-18 09:55:51 [SUCCESS]: [✔] Uploaded domain controller (DC) configuration files 2021-02-18 09:55:51 [ INFO]: [ ] Uploading Windows package installers to blob storage AccountName: shmtestbartifactsqtxiyzd, ContainerName: sre-rds-sh-packages Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted VersionId ---- -------- ------ ----------- ------------ ---------- ------------ --------- --------- GoogleChrome_x64.msi BlockBlob -1 2021-02-18 09:55:52Z False PuTTY_x64.msi BlockBlob -1 2021-02-18 09:55:53Z False 2021-02-18 09:55:53 [SUCCESS]: [✔] Uploaded Windows package installers 2021-02-18 09:55:53 [ INFO]: Ensuring that resource group 'RG_SHM_TESTB_DC' exists... 2021-02-18 09:55:54 [ INFO]: [ ] Creating resource group 'RG_SHM_TESTB_DC' 2021-02-18 09:55:57 [SUCCESS]: [✔] Created resource group 'RG_SHM_TESTB_DC' 2021-02-18 09:55:57 [ INFO]: Creating/retrieving secrets from Key Vault 'kv-shm-testb'... WARNING: Upcoming breaking changes in the cmdlet 'Get-AzKeyVaultSecret' : - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSDeletedKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell. WARNING: Upcoming breaking changes in the cmdlet 'Get-AzKeyVaultSecret' : - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSDeletedKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell. WARNING: Upcoming breaking changes in the cmdlet 'Get-AzKeyVaultSecret' : - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSDeletedKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell. WARNING: Upcoming breaking changes in the cmdlet 'Get-AzKeyVaultSecret' : - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSDeletedKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell. WARNING: Upcoming breaking changes in the cmdlet 'Get-AzKeyVaultSecret' : - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSDeletedKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell. WARNING: Upcoming breaking changes in the cmdlet 'Get-AzKeyVaultSecret' : - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' - The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSDeletedKeyVaultSecret' is changing - The following properties in the output type are being deprecated : 'SecretValueText' - The change is expected to take effect from the version : '3.0.0' Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell. 2021-02-18 09:56:05 [ INFO]: Deploying domain controller (DC) from template... VERBOSE: Performing the operation "Creating Deployment" on target "RG_SHM_TESTB_DC". WARNING: The DeploymentDebug setting has been enabled. This can potentially log secrets like passwords used in resource property or listKeys operations when you retrieve the deployment operations through Get-AzResourceGroupDeploymentOperation VERBOSE: 09:56:10 - Template is valid. VERBOSE: 09:56:11 - Create template deployment 'shm-dc-template' VERBOSE: 09:56:11 - Checking deployment status in 5 seconds VERBOSE: 09:56:16 - Resource Microsoft.Compute/virtualMachines 'DC1-SHM-TESTB' provisioning status is running VERBOSE: 09:56:16 - Resource Microsoft.Compute/virtualMachines 'DC2-SHM-TESTB' provisioning status is running VERBOSE: 09:56:16 - Resource Microsoft.Network/networkInterfaces 'DC2-SHM-TESTB-NIC' provisioning status is succeeded VERBOSE: 09:56:16 - Resource Microsoft.Network/networkInterfaces 'DC1-SHM-TESTB-NIC' provisioning status is succeeded VERBOSE: 09:56:16 - Resource Microsoft.Compute/availabilitySets 'AVSET-SHM-TESTB-VM-DC' provisioning status is succeeded VERBOSE: 09:56:16 - Checking deployment status in 13 seconds VERBOSE: 09:56:30 - Checking deployment status in 5 seconds VERBOSE: 09:56:35 - Checking deployment status in 5 seconds VERBOSE: 09:56:40 - Checking deployment status in 5 seconds VERBOSE: 09:56:45 - Checking deployment status in 5 seconds VERBOSE: 09:56:51 - Checking deployment status in 5 seconds VERBOSE: 09:56:57 - Checking deployment status in 5 seconds VERBOSE: 09:57:02 - Checking deployment status in 5 seconds VERBOSE: 09:57:07 - Checking deployment status in 5 seconds VERBOSE: 09:57:13 - Checking deployment status in 5 seconds VERBOSE: 09:57:18 - Checking deployment status in 5 seconds VERBOSE: 09:57:23 - Checking deployment status in 5 seconds VERBOSE: 09:57:29 - Checking deployment status in 5 seconds VERBOSE: 09:57:34 - Checking deployment status in 5 seconds VERBOSE: 09:57:40 - Checking deployment status in 5 seconds VERBOSE: 09:57:45 - Checking deployment status in 5 seconds VERBOSE: 09:57:50 - Checking deployment status in 5 seconds VERBOSE: 09:57:55 - Checking deployment status in 5 seconds VERBOSE: 09:58:01 - Checking deployment status in 5 seconds VERBOSE: 09:58:06 - Checking deployment status in 5 seconds VERBOSE: 09:58:12 - Checking deployment status in 5 seconds VERBOSE: 09:58:17 - Checking deployment status in 13 seconds VERBOSE: 09:58:30 - Resource Microsoft.Compute/virtualMachines/extensions 'DC2-SHM-TESTB/bginfo' provisioning status is running VERBOSE: 09:58:30 - Resource Microsoft.Compute/virtualMachines 'DC2-SHM-TESTB' provisioning status is succeeded VERBOSE: 09:58:30 - Checking deployment status in 7 seconds VERBOSE: 09:58:37 - Checking deployment status in 5 seconds VERBOSE: 09:58:43 - Checking deployment status in 5 seconds VERBOSE: 09:58:48 - Checking deployment status in 5 seconds VERBOSE: 09:58:53 - Checking deployment status in 5 seconds VERBOSE: 09:58:59 - Checking deployment status in 5 seconds VERBOSE: 09:59:04 - Checking deployment status in 5 seconds VERBOSE: 09:59:10 - Checking deployment status in 5 seconds VERBOSE: 09:59:15 - Checking deployment status in 5 seconds VERBOSE: 09:59:21 - Checking deployment status in 5 seconds VERBOSE: 09:59:26 - Checking deployment status in 5 seconds VERBOSE: 09:59:32 - Checking deployment status in 5 seconds VERBOSE: 09:59:37 - Checking deployment status in 5 seconds VERBOSE: 09:59:42 - Checking deployment status in 5 seconds VERBOSE: 09:59:48 - Checking deployment status in 5 seconds VERBOSE: 09:59:54 - Checking deployment status in 5 seconds VERBOSE: 09:59:59 - Checking deployment status in 5 seconds VERBOSE: 10:00:04 - Checking deployment status in 5 seconds VERBOSE: 10:00:09 - Checking deployment status in 5 seconds VERBOSE: 10:00:15 - Checking deployment status in 16 seconds VERBOSE: 10:00:31 - Checking deployment status in 6 seconds VERBOSE: 10:00:38 - Checking deployment status in 5 seconds VERBOSE: 10:00:43 - Checking deployment status in 5 seconds VERBOSE: 10:00:48 - Resource Microsoft.Compute/virtualMachines/extensions 'DC2-SHM-TESTB/bginfo' provisioning status is succeeded VERBOSE: 10:00:49 - Checking deployment status in 5 seconds VERBOSE: 10:00:54 - Checking deployment status in 5 seconds VERBOSE: 10:01:00 - Checking deployment status in 5 seconds VERBOSE: 10:01:05 - Checking deployment status in 5 seconds VERBOSE: 10:01:11 - Checking deployment status in 5 seconds VERBOSE: 10:01:16 - Checking deployment status in 5 seconds VERBOSE: 10:01:22 - Checking deployment status in 5 seconds VERBOSE: 10:01:27 - Checking deployment status in 5 seconds VERBOSE: 10:01:32 - Checking deployment status in 5 seconds VERBOSE: 10:01:38 - Checking deployment status in 5 seconds VERBOSE: 10:01:43 - Checking deployment status in 5 seconds VERBOSE: 10:01:49 - Checking deployment status in 5 seconds VERBOSE: 10:01:54 - Checking deployment status in 5 seconds VERBOSE: 10:01:59 - Checking deployment status in 5 seconds VERBOSE: 10:02:04 - Checking deployment status in 5 seconds VERBOSE: 10:02:09 - Checking deployment status in 5 seconds VERBOSE: 10:02:14 - Checking deployment status in 5 seconds VERBOSE: 10:02:20 - Checking deployment status in 11 seconds VERBOSE: 10:02:31 - Checking deployment status in 5 seconds VERBOSE: 10:02:36 - Checking deployment status in 5 seconds VERBOSE: 10:02:42 - Checking deployment status in 5 seconds VERBOSE: 10:02:47 - Checking deployment status in 5 seconds VERBOSE: 10:02:52 - Checking deployment status in 5 seconds VERBOSE: 10:02:57 - Checking deployment status in 5 seconds VERBOSE: 10:03:03 - Checking deployment status in 5 seconds VERBOSE: 10:03:08 - Checking deployment status in 5 seconds VERBOSE: 10:03:14 - Checking deployment status in 5 seconds VERBOSE: 10:03:19 - Checking deployment status in 5 seconds VERBOSE: 10:03:25 - Checking deployment status in 5 seconds VERBOSE: 10:03:30 - Checking deployment status in 5 seconds VERBOSE: 10:03:35 - Checking deployment status in 5 seconds VERBOSE: 10:03:40 - Checking deployment status in 5 seconds VERBOSE: 10:03:46 - Checking deployment status in 5 seconds VERBOSE: 10:03:51 - Checking deployment status in 5 seconds VERBOSE: 10:03:56 - Checking deployment status in 5 seconds VERBOSE: 10:04:01 - Checking deployment status in 5 seconds VERBOSE: 10:04:07 - Checking deployment status in 5 seconds VERBOSE: 10:04:12 - Checking deployment status in 5 seconds VERBOSE: 10:04:17 - Checking deployment status in 15 seconds VERBOSE: 10:04:32 - Checking deployment status in 5 seconds VERBOSE: 10:04:37 - Checking deployment status in 5 seconds VERBOSE: 10:04:42 - Checking deployment status in 5 seconds VERBOSE: 10:04:47 - Checking deployment status in 5 seconds VERBOSE: 10:04:53 - Checking deployment status in 5 seconds VERBOSE: 10:04:58 - Checking deployment status in 5 seconds VERBOSE: 10:05:04 - Checking deployment status in 5 seconds VERBOSE: 10:05:09 - Checking deployment status in 5 seconds VERBOSE: 10:05:14 - Checking deployment status in 5 seconds VERBOSE: 10:05:20 - Checking deployment status in 5 seconds VERBOSE: 10:05:25 - Checking deployment status in 5 seconds VERBOSE: 10:05:31 - Checking deployment status in 5 seconds VERBOSE: 10:05:36 - Checking deployment status in 5 seconds VERBOSE: 10:05:42 - Checking deployment status in 5 seconds VERBOSE: 10:05:47 - Checking deployment status in 5 seconds VERBOSE: 10:05:52 - Checking deployment status in 5 seconds VERBOSE: 10:05:58 - Checking deployment status in 5 seconds VERBOSE: 10:06:03 - Checking deployment status in 5 seconds VERBOSE: 10:06:09 - Checking deployment status in 5 seconds VERBOSE: 10:06:14 - Checking deployment status in 5 seconds VERBOSE: 10:06:19 - Checking deployment status in 13 seconds VERBOSE: 10:06:32 - Checking deployment status in 5 seconds VERBOSE: 10:06:38 - Checking deployment status in 5 seconds VERBOSE: 10:06:43 - Checking deployment status in 5 seconds VERBOSE: 10:06:49 - Checking deployment status in 5 seconds VERBOSE: 10:06:54 - Checking deployment status in 5 seconds VERBOSE: 10:06:59 - Checking deployment status in 5 seconds VERBOSE: 10:07:04 - Checking deployment status in 5 seconds VERBOSE: 10:07:10 - Checking deployment status in 5 seconds VERBOSE: 10:07:15 - Checking deployment status in 5 seconds VERBOSE: 10:07:21 - Checking deployment status in 5 seconds VERBOSE: 10:07:26 - Checking deployment status in 5 seconds VERBOSE: 10:07:31 - Checking deployment status in 5 seconds VERBOSE: 10:07:36 - Checking deployment status in 5 seconds VERBOSE: 10:07:42 - Checking deployment status in 5 seconds VERBOSE: 10:07:47 - Checking deployment status in 5 seconds VERBOSE: 10:07:52 - Checking deployment status in 5 seconds VERBOSE: 10:07:57 - Checking deployment status in 5 seconds VERBOSE: 10:08:03 - Checking deployment status in 5 seconds VERBOSE: 10:08:08 - Checking deployment status in 5 seconds VERBOSE: 10:08:13 - Checking deployment status in 5 seconds VERBOSE: 10:08:18 - Checking deployment status in 15 seconds VERBOSE: 10:08:33 - Checking deployment status in 5 seconds VERBOSE: 10:08:38 - Checking deployment status in 5 seconds VERBOSE: 10:08:44 - Checking deployment status in 5 seconds VERBOSE: 10:08:49 - Checking deployment status in 5 seconds VERBOSE: 10:08:54 - Checking deployment status in 5 seconds VERBOSE: 10:09:00 - Checking deployment status in 5 seconds VERBOSE: 10:09:05 - Checking deployment status in 5 seconds VERBOSE: 10:09:10 - Checking deployment status in 5 seconds VERBOSE: 10:09:15 - Checking deployment status in 5 seconds VERBOSE: 10:09:20 - Checking deployment status in 5 seconds VERBOSE: 10:09:26 - Checking deployment status in 5 seconds VERBOSE: 10:09:31 - Checking deployment status in 5 seconds VERBOSE: 10:09:36 - Checking deployment status in 5 seconds VERBOSE: 10:09:42 - Checking deployment status in 5 seconds VERBOSE: 10:09:47 - Checking deployment status in 5 seconds VERBOSE: 10:09:52 - Checking deployment status in 5 seconds VERBOSE: 10:09:58 - Checking deployment status in 5 seconds VERBOSE: 10:10:04 - Checking deployment status in 5 seconds VERBOSE: 10:10:09 - Checking deployment status in 5 seconds VERBOSE: 10:10:14 - Checking deployment status in 5 seconds VERBOSE: 10:10:20 - Checking deployment status in 15 seconds VERBOSE: 10:10:35 - Checking deployment status in 5 seconds VERBOSE: 10:10:40 - Checking deployment status in 5 seconds VERBOSE: 10:10:45 - Checking deployment status in 5 seconds VERBOSE: 10:10:51 - Checking deployment status in 5 seconds VERBOSE: 10:10:56 - Checking deployment status in 5 seconds VERBOSE: 10:11:01 - Checking deployment status in 5 seconds VERBOSE: 10:11:07 - Checking deployment status in 5 seconds VERBOSE: 10:11:12 - Checking deployment status in 5 seconds VERBOSE: 10:11:18 - Checking deployment status in 5 seconds VERBOSE: 10:11:23 - Checking deployment status in 5 seconds VERBOSE: 10:11:29 - Checking deployment status in 5 seconds VERBOSE: 10:11:34 - Checking deployment status in 5 seconds VERBOSE: 10:11:40 - Checking deployment status in 5 seconds VERBOSE: 10:11:45 - Checking deployment status in 5 seconds VERBOSE: 10:11:50 - Checking deployment status in 5 seconds VERBOSE: 10:11:56 - Checking deployment status in 5 seconds VERBOSE: 10:12:01 - Checking deployment status in 5 seconds VERBOSE: 10:12:06 - Checking deployment status in 5 seconds VERBOSE: 10:12:12 - Checking deployment status in 5 seconds VERBOSE: 10:12:17 - Checking deployment status in 5 seconds VERBOSE: 10:12:23 - Checking deployment status in 13 seconds VERBOSE: 10:12:36 - Checking deployment status in 5 seconds VERBOSE: 10:12:41 - Resource Microsoft.Compute/virtualMachines/extensions 'DC1-SHM-TESTB/CreateADForest' provisioning status is running VERBOSE: 10:12:41 - Resource Microsoft.Compute/virtualMachines/extensions 'DC1-SHM-TESTB/bginfo' provisioning status is running VERBOSE: 10:12:41 - Resource Microsoft.Compute/virtualMachines 'DC1-SHM-TESTB' provisioning status is succeeded VERBOSE: 10:12:41 - Checking deployment status in 12 seconds VERBOSE: 10:12:54 - Checking deployment status in 5 seconds VERBOSE: 10:12:59 - Checking deployment status in 5 seconds VERBOSE: 10:13:05 - Checking deployment status in 5 seconds VERBOSE: 10:13:10 - Checking deployment status in 5 seconds VERBOSE: 10:13:15 - Checking deployment status in 5 seconds VERBOSE: 10:13:21 - Checking deployment status in 5 seconds VERBOSE: 10:13:26 - Checking deployment status in 5 seconds VERBOSE: 10:13:31 - Checking deployment status in 5 seconds VERBOSE: 10:13:37 - Checking deployment status in 5 seconds VERBOSE: 10:13:42 - Checking deployment status in 5 seconds VERBOSE: 10:13:48 - Checking deployment status in 5 seconds VERBOSE: 10:13:53 - Checking deployment status in 5 seconds VERBOSE: 10:13:58 - Checking deployment status in 5 seconds VERBOSE: 10:14:04 - Checking deployment status in 5 seconds VERBOSE: 10:14:09 - Checking deployment status in 5 seconds VERBOSE: 10:14:15 - Checking deployment status in 5 seconds VERBOSE: 10:14:20 - Checking deployment status in 5 seconds VERBOSE: 10:14:25 - Checking deployment status in 5 seconds VERBOSE: 10:14:31 - Checking deployment status in 5 seconds VERBOSE: 10:14:36 - Checking deployment status in 5 seconds VERBOSE: 10:14:41 - Checking deployment status in 12 seconds VERBOSE: 10:14:54 - Checking deployment status in 5 seconds VERBOSE: 10:14:59 - Checking deployment status in 5 seconds VERBOSE: 10:15:04 - Checking deployment status in 5 seconds VERBOSE: 10:15:09 - Checking deployment status in 5 seconds VERBOSE: 10:15:14 - Checking deployment status in 5 seconds VERBOSE: 10:15:20 - Checking deployment status in 5 seconds VERBOSE: 10:15:25 - Checking deployment status in 5 seconds VERBOSE: 10:15:31 - Checking deployment status in 5 seconds VERBOSE: 10:15:36 - Checking deployment status in 5 seconds VERBOSE: 10:15:41 - Checking deployment status in 5 seconds VERBOSE: 10:15:47 - Checking deployment status in 5 seconds VERBOSE: 10:15:52 - Checking deployment status in 5 seconds VERBOSE: 10:15:57 - Checking deployment status in 5 seconds VERBOSE: 10:16:03 - Checking deployment status in 5 seconds VERBOSE: 10:16:09 - Checking deployment status in 5 seconds VERBOSE: 10:16:14 - Checking deployment status in 5 seconds VERBOSE: 10:16:19 - Checking deployment status in 5 seconds VERBOSE: 10:16:25 - Checking deployment status in 5 seconds VERBOSE: 10:16:30 - Checking deployment status in 5 seconds VERBOSE: 10:16:36 - Checking deployment status in 5 seconds VERBOSE: 10:16:41 - Checking deployment status in 13 seconds VERBOSE: 10:16:54 - Checking deployment status in 5 seconds VERBOSE: 10:17:00 - Checking deployment status in 5 seconds VERBOSE: 10:17:05 - Checking deployment status in 5 seconds VERBOSE: 10:17:10 - Checking deployment status in 5 seconds VERBOSE: 10:17:16 - Checking deployment status in 5 seconds VERBOSE: 10:17:21 - Checking deployment status in 5 seconds VERBOSE: 10:17:26 - Checking deployment status in 5 seconds VERBOSE: 10:17:32 - Checking deployment status in 5 seconds VERBOSE: 10:17:37 - Checking deployment status in 5 seconds VERBOSE: 10:17:42 - Checking deployment status in 5 seconds VERBOSE: 10:17:47 - Checking deployment status in 5 seconds VERBOSE: 10:17:53 - Checking deployment status in 5 seconds VERBOSE: 10:17:58 - Checking deployment status in 5 seconds VERBOSE: 10:18:03 - Checking deployment status in 5 seconds VERBOSE: 10:18:09 - Checking deployment status in 5 seconds VERBOSE: 10:18:14 - Checking deployment status in 5 seconds VERBOSE: 10:18:19 - Checking deployment status in 5 seconds VERBOSE: 10:18:25 - Checking deployment status in 5 seconds VERBOSE: 10:18:30 - Checking deployment status in 5 seconds VERBOSE: 10:18:35 - Checking deployment status in 5 seconds VERBOSE: 10:18:41 - Checking deployment status in 14 seconds VERBOSE: 10:18:56 - Checking deployment status in 5 seconds VERBOSE: 10:19:01 - Checking deployment status in 5 seconds VERBOSE: 10:19:06 - Checking deployment status in 5 seconds VERBOSE: 10:19:11 - Checking deployment status in 5 seconds VERBOSE: 10:19:17 - Checking deployment status in 5 seconds VERBOSE: 10:19:22 - Checking deployment status in 5 seconds VERBOSE: 10:19:27 - Checking deployment status in 5 seconds VERBOSE: 10:19:33 - Checking deployment status in 5 seconds VERBOSE: 10:19:38 - Checking deployment status in 5 seconds VERBOSE: 10:19:43 - Checking deployment status in 5 seconds VERBOSE: 10:19:48 - Checking deployment status in 5 seconds VERBOSE: 10:19:54 - Checking deployment status in 5 seconds VERBOSE: 10:19:59 - Checking deployment status in 5 seconds VERBOSE: 10:20:04 - Checking deployment status in 5 seconds VERBOSE: 10:20:10 - Checking deployment status in 5 seconds VERBOSE: 10:20:15 - Checking deployment status in 5 seconds VERBOSE: 10:20:20 - Checking deployment status in 5 seconds VERBOSE: 10:20:25 - Checking deployment status in 5 seconds VERBOSE: 10:20:31 - Checking deployment status in 5 seconds VERBOSE: 10:20:36 - Checking deployment status in 5 seconds VERBOSE: 10:20:41 - Checking deployment status in 14 seconds VERBOSE: 10:20:56 - Checking deployment status in 5 seconds VERBOSE: 10:21:01 - Checking deployment status in 5 seconds VERBOSE: 10:21:06 - Checking deployment status in 5 seconds VERBOSE: 10:21:12 - Checking deployment status in 5 seconds VERBOSE: 10:21:17 - Checking deployment status in 5 seconds VERBOSE: 10:21:23 - Checking deployment status in 5 seconds VERBOSE: 10:21:28 - Checking deployment status in 5 seconds VERBOSE: 10:21:33 - Checking deployment status in 5 seconds VERBOSE: 10:21:39 - Checking deployment status in 5 seconds VERBOSE: 10:21:44 - Checking deployment status in 5 seconds VERBOSE: 10:21:49 - Checking deployment status in 5 seconds VERBOSE: 10:21:55 - Checking deployment status in 5 seconds VERBOSE: 10:22:01 - Checking deployment status in 5 seconds VERBOSE: 10:22:06 - Checking deployment status in 5 seconds VERBOSE: 10:22:11 - Checking deployment status in 5 seconds VERBOSE: 10:22:17 - Checking deployment status in 5 seconds VERBOSE: 10:22:22 - Checking deployment status in 5 seconds VERBOSE: 10:22:27 - Checking deployment status in 5 seconds VERBOSE: 10:22:33 - Checking deployment status in 5 seconds VERBOSE: 10:22:38 - Checking deployment status in 5 seconds VERBOSE: 10:22:43 - Checking deployment status in 13 seconds VERBOSE: 10:22:57 - Checking deployment status in 5 seconds VERBOSE: 10:23:03 - Checking deployment status in 5 seconds VERBOSE: 10:23:08 - Checking deployment status in 5 seconds VERBOSE: 10:23:13 - Checking deployment status in 5 seconds VERBOSE: 10:23:19 - Checking deployment status in 5 seconds VERBOSE: 10:23:25 - Checking deployment status in 5 seconds VERBOSE: 10:23:30 - Checking deployment status in 5 seconds VERBOSE: 10:23:35 - Checking deployment status in 5 seconds VERBOSE: 10:23:40 - Checking deployment status in 5 seconds VERBOSE: 10:23:46 - Checking deployment status in 5 seconds VERBOSE: 10:23:51 - Checking deployment status in 5 seconds VERBOSE: 10:23:56 - Checking deployment status in 5 seconds VERBOSE: 10:24:02 - Checking deployment status in 5 seconds VERBOSE: 10:24:07 - Checking deployment status in 5 seconds VERBOSE: 10:24:12 - Checking deployment status in 5 seconds VERBOSE: 10:24:17 - Checking deployment status in 5 seconds VERBOSE: 10:24:23 - Checking deployment status in 5 seconds VERBOSE: 10:24:29 - Checking deployment status in 5 seconds VERBOSE: 10:24:34 - Checking deployment status in 5 seconds VERBOSE: 10:24:39 - Checking deployment status in 5 seconds VERBOSE: 10:24:45 - Checking deployment status in 13 seconds VERBOSE: 10:24:58 - Checking deployment status in 5 seconds VERBOSE: 10:25:03 - Checking deployment status in 5 seconds VERBOSE: 10:25:08 - Checking deployment status in 5 seconds VERBOSE: 10:25:14 - Checking deployment status in 5 seconds VERBOSE: 10:25:19 - Checking deployment status in 5 seconds VERBOSE: 10:25:25 - Checking deployment status in 5 seconds VERBOSE: 10:25:30 - Checking deployment status in 5 seconds VERBOSE: 10:25:36 - Checking deployment status in 5 seconds VERBOSE: 10:25:42 - Checking deployment status in 5 seconds VERBOSE: 10:25:47 - Checking deployment status in 5 seconds VERBOSE: 10:25:52 - Checking deployment status in 5 seconds VERBOSE: 10:25:57 - Checking deployment status in 5 seconds VERBOSE: 10:26:03 - Checking deployment status in 5 seconds VERBOSE: 10:26:08 - Checking deployment status in 5 seconds VERBOSE: 10:26:13 - Checking deployment status in 5 seconds VERBOSE: 10:26:19 - Checking deployment status in 5 seconds VERBOSE: 10:26:24 - Checking deployment status in 5 seconds VERBOSE: 10:26:30 - Checking deployment status in 5 seconds VERBOSE: 10:26:35 - Checking deployment status in 5 seconds VERBOSE: 10:26:40 - Checking deployment status in 16 seconds VERBOSE: 10:26:56 - Checking deployment status in 5 seconds VERBOSE: 10:27:01 - Checking deployment status in 5 seconds VERBOSE: 10:27:07 - Checking deployment status in 5 seconds VERBOSE: 10:27:12 - Checking deployment status in 5 seconds VERBOSE: 10:27:17 - Checking deployment status in 5 seconds VERBOSE: 10:27:23 - Checking deployment status in 5 seconds VERBOSE: 10:27:28 - Checking deployment status in 5 seconds VERBOSE: 10:27:33 - Checking deployment status in 5 seconds VERBOSE: 10:27:38 - Checking deployment status in 5 seconds VERBOSE: 10:27:44 - Checking deployment status in 5 seconds VERBOSE: 10:27:49 - Checking deployment status in 5 seconds VERBOSE: 10:27:55 - Checking deployment status in 5 seconds VERBOSE: 10:28:00 - Checking deployment status in 5 seconds VERBOSE: 10:28:05 - Checking deployment status in 5 seconds VERBOSE: 10:28:11 - Checking deployment status in 5 seconds VERBOSE: 10:28:16 - Checking deployment status in 5 seconds VERBOSE: 10:28:21 - Checking deployment status in 5 seconds VERBOSE: 10:28:26 - Checking deployment status in 5 seconds VERBOSE: 10:28:32 - Checking deployment status in 5 seconds VERBOSE: 10:28:37 - Checking deployment status in 5 seconds VERBOSE: 10:28:42 - Checking deployment status in 16 seconds VERBOSE: 10:28:59 - Checking deployment status in 5 seconds VERBOSE: 10:29:04 - Checking deployment status in 5 seconds VERBOSE: 10:29:10 - Checking deployment status in 5 seconds VERBOSE: 10:29:15 - Checking deployment status in 5 seconds VERBOSE: 10:29:20 - Checking deployment status in 5 seconds VERBOSE: 10:29:26 - Checking deployment status in 5 seconds VERBOSE: 10:29:31 - Checking deployment status in 5 seconds VERBOSE: 10:29:37 - Checking deployment status in 5 seconds VERBOSE: 10:29:42 - Checking deployment status in 5 seconds VERBOSE: 10:29:47 - Checking deployment status in 5 seconds VERBOSE: 10:29:53 - Checking deployment status in 5 seconds VERBOSE: 10:29:58 - Checking deployment status in 5 seconds VERBOSE: 10:30:03 - Checking deployment status in 5 seconds VERBOSE: 10:30:08 - Checking deployment status in 5 seconds VERBOSE: 10:30:14 - Checking deployment status in 5 seconds VERBOSE: 10:30:19 - Checking deployment status in 5 seconds VERBOSE: 10:30:24 - Checking deployment status in 5 seconds VERBOSE: 10:30:30 - Checking deployment status in 5 seconds VERBOSE: 10:30:35 - Resource Microsoft.Compute/virtualMachines/extensions 'DC2-SHM-TESTB/CreateADBDC' provisioning status is running VERBOSE: 10:30:35 - Resource Microsoft.Compute/virtualMachines/extensions 'DC1-SHM-TESTB/CreateADForest' provisioning status is succeeded VERBOSE: 10:30:35 - Resource Microsoft.Compute/virtualMachines/extensions 'DC1-SHM-TESTB/bginfo' provisioning status is succeeded VERBOSE: 10:30:35 - Checking deployment status in 13 seconds VERBOSE: 10:30:48 - Checking deployment status in 5 seconds VERBOSE: 10:30:53 - Checking deployment status in 5 seconds VERBOSE: 10:30:59 - Checking deployment status in 5 seconds VERBOSE: 10:31:04 - Checking deployment status in 5 seconds VERBOSE: 10:31:10 - Checking deployment status in 5 seconds VERBOSE: 10:31:15 - Checking deployment status in 5 seconds VERBOSE: 10:31:20 - Checking deployment status in 5 seconds VERBOSE: 10:31:26 - Checking deployment status in 5 seconds VERBOSE: 10:31:31 - Checking deployment status in 5 seconds VERBOSE: 10:31:36 - Checking deployment status in 5 seconds VERBOSE: 10:31:42 - Checking deployment status in 5 seconds VERBOSE: 10:31:47 - Checking deployment status in 5 seconds VERBOSE: 10:31:53 - Checking deployment status in 5 seconds VERBOSE: 10:31:58 - Checking deployment status in 5 seconds VERBOSE: 10:32:03 - Checking deployment status in 5 seconds VERBOSE: 10:32:09 - Checking deployment status in 5 seconds VERBOSE: 10:32:14 - Checking deployment status in 5 seconds VERBOSE: 10:32:20 - Checking deployment status in 5 seconds VERBOSE: 10:32:25 - Checking deployment status in 5 seconds VERBOSE: 10:32:30 - Checking deployment status in 5 seconds VERBOSE: 10:32:36 - Checking deployment status in 13 seconds VERBOSE: 10:32:49 - Checking deployment status in 5 seconds VERBOSE: 10:32:54 - Checking deployment status in 5 seconds VERBOSE: 10:32:59 - Checking deployment status in 5 seconds VERBOSE: 10:33:05 - Checking deployment status in 5 seconds VERBOSE: 10:33:10 - Checking deployment status in 5 seconds VERBOSE: 10:33:15 - Checking deployment status in 5 seconds VERBOSE: 10:33:20 - Checking deployment status in 5 seconds VERBOSE: 10:33:26 - Checking deployment status in 5 seconds VERBOSE: 10:33:31 - Checking deployment status in 5 seconds VERBOSE: 10:33:37 - Checking deployment status in 5 seconds VERBOSE: 10:33:42 - Checking deployment status in 5 seconds VERBOSE: 10:33:47 - Checking deployment status in 5 seconds VERBOSE: 10:33:53 - Checking deployment status in 5 seconds VERBOSE: 10:33:58 - Checking deployment status in 5 seconds VERBOSE: 10:34:03 - Checking deployment status in 5 seconds VERBOSE: 10:34:09 - Checking deployment status in 5 seconds VERBOSE: 10:34:14 - Checking deployment status in 5 seconds VERBOSE: 10:34:19 - Checking deployment status in 5 seconds VERBOSE: 10:34:24 - Checking deployment status in 5 seconds VERBOSE: 10:34:30 - Checking deployment status in 5 seconds VERBOSE: 10:34:35 - Checking deployment status in 15 seconds VERBOSE: 10:34:51 - Checking deployment status in 5 seconds VERBOSE: 10:34:56 - Checking deployment status in 5 seconds VERBOSE: 10:35:01 - Checking deployment status in 5 seconds VERBOSE: 10:35:07 - Checking deployment status in 5 seconds VERBOSE: 10:35:12 - Checking deployment status in 5 seconds VERBOSE: 10:35:17 - Checking deployment status in 5 seconds VERBOSE: 10:35:23 - Checking deployment status in 5 seconds VERBOSE: 10:35:28 - Checking deployment status in 5 seconds VERBOSE: 10:35:33 - Checking deployment status in 5 seconds VERBOSE: 10:35:38 - Checking deployment status in 5 seconds VERBOSE: 10:35:44 - Checking deployment status in 5 seconds VERBOSE: 10:35:49 - Checking deployment status in 5 seconds VERBOSE: 10:35:55 - Checking deployment status in 5 seconds VERBOSE: 10:36:00 - Checking deployment status in 5 seconds VERBOSE: 10:36:05 - Checking deployment status in 5 seconds VERBOSE: 10:36:10 - Checking deployment status in 5 seconds VERBOSE: 10:36:16 - Checking deployment status in 5 seconds VERBOSE: 10:36:21 - Checking deployment status in 5 seconds VERBOSE: 10:36:26 - Checking deployment status in 5 seconds VERBOSE: 10:36:32 - Checking deployment status in 5 seconds VERBOSE: 10:36:37 - Checking deployment status in 14 seconds VERBOSE: 10:36:51 - Checking deployment status in 5 seconds VERBOSE: 10:36:57 - Checking deployment status in 5 seconds VERBOSE: 10:37:02 - Checking deployment status in 5 seconds VERBOSE: 10:37:07 - Checking deployment status in 5 seconds VERBOSE: 10:37:13 - Checking deployment status in 5 seconds VERBOSE: 10:37:18 - Checking deployment status in 5 seconds VERBOSE: 10:37:23 - Checking deployment status in 5 seconds VERBOSE: 10:37:29 - Checking deployment status in 5 seconds VERBOSE: 10:37:34 - Checking deployment status in 5 seconds VERBOSE: 10:37:39 - Checking deployment status in 5 seconds VERBOSE: 10:37:44 - Checking deployment status in 5 seconds VERBOSE: 10:37:50 - Checking deployment status in 5 seconds VERBOSE: 10:37:55 - Checking deployment status in 5 seconds VERBOSE: 10:38:00 - Checking deployment status in 5 seconds VERBOSE: 10:38:06 - Checking deployment status in 5 seconds VERBOSE: 10:38:11 - Checking deployment status in 5 seconds VERBOSE: 10:38:16 - Checking deployment status in 5 seconds VERBOSE: 10:38:21 - Checking deployment status in 5 seconds VERBOSE: 10:38:27 - Checking deployment status in 5 seconds VERBOSE: 10:38:32 - Checking deployment status in 5 seconds VERBOSE: 10:38:37 - Checking deployment status in 16 seconds VERBOSE: 10:38:54 - Resource Microsoft.Compute/virtualMachines/extensions 'DC2-SHM-TESTB/CreateADBDC' provisioning status is succeeded ResourceGroupName : RG_SHM_TESTB_DC OnErrorDeployment : DeploymentName : shm-dc-template CorrelationId : 849c16a4-504c-4709-870b-f790a2937b1f ProvisioningState : Succeeded Timestamp : 18/02/2021 10:38:45 Mode : Incremental TemplateLink : TemplateLinkString : DeploymentDebugLogLevel : ResponseContent Parameters : {[administrator_Password, Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.DeploymentVariable], [administrator_User, Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.DeploymentVariable], [artifacts_Location, Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.DeploymentVariable], [artifacts_Location_SAS_Token, Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.DeploymentVariable]…} Tags : ParametersString : Name Type Value ================================ ========================= ========== administrator_Password SecureString administrator_User String domaintestbadmin artifacts_Location String https://shmtestbartifactsqtxiyzd.blob.core.windows.net artifacts_Location_SAS_Token SecureString bootDiagnostics_Account_Name String shmtestbbootdiagsqtxiyzd dC1_Data_Disk_Size_GB Int 20 dC1_Data_Disk_Type String Standard_LRS dC1_Host_Name String DC1-SHM-TESTB dC1_IP_Address String 10.0.0.4 dC1_Os_Disk_Size_GB Int 128 dC1_Os_Disk_Type String Standard_LRS dC1_VM_Name String DC1-SHM-TESTB dC1_VM_Size String Standard_D2s_v3 dC2_Data_Disk_Size_GB Int 20 dC2_Data_Disk_Type String Standard_LRS dC2_Host_Name String DC2-SHM-TESTB dC2_IP_Address String 10.0.0.5 dC2_Os_Disk_Size_GB Int 128 dC2_Os_Disk_Type String Standard_LRS dC2_VM_Name String DC2-SHM-TESTB dC2_VM_Size String Standard_D2s_v3 domain_Name String testb.dsgroupdev.co.uk domain_NetBIOS_Name String TESTB external_DNS_Resolver String 168.63.129.16 safeMode_Password SecureString shm_Id String testb virtual_Network_Name String VNET_SHM_TESTB virtual_Network_Resource_Group String RG_SHM_TESTB_NETWORKING virtual_Network_Subnet String IdentitySubnet Outputs : OutputsString : WARNING: Upcoming breaking changes in the cmdlet 'Get-AzResourceGroupDeploymentOperation' : - The output type is changing from the existing type :'System.Management.Automation.PSObject' to the new type :'PSDeploymentOperation' Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell. 2021-02-18 10:38:56 [ INFO]: CreateADBDC: ProvisioningState/succeeded DSC configuration was applied successfully. 2021-02-18 10:38:56 [ INFO]: CreateADBDC: ComponentStatus/DscConfigurationLog/succeeded [2021-02-18 10:38:35Z] [WARNING] [DC2-SHM-TESTB]: [[xPendingReboot]RebootAfterPromotion] Unable to query CCM_ClientUtilities: Invalid namespace [2021-02-18 10:38:35Z] [VERBOSE] [DC2-SHM-TESTB]: [[xPendingReboot]RebootAfterPromotion] No pending reboots found. [2021-02-18 10:38:35Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ End Test ] [[xPendingReboot]RebootAfterPromotion] in 0.0940 seconds. [2021-02-18 10:38:35Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ Skip Set ] [[xPendingReboot]RebootAfterPromotion] [2021-02-18 10:38:35Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ End Resource ] [[xPendingReboot]RebootAfterPromotion] [2021-02-18 10:38:35Z] [VERBOSE] [DC2-SHM-TESTB]: [] Consistency check completed. [2021-02-18 10:38:35Z] [VERBOSE] Operation 'Invoke CimMethod' complete. [2021-02-18 10:38:35Z] [VERBOSE] Time taken for configuration job to complete is 26.832 seconds [2021-02-18 10:38:38Z] [VERBOSE] Performing the operation "Start-DscConfiguration: SendMetaConfigurationApply" on target "MSFT_DSCLocalConfigurationManager". [2021-02-18 10:38:38Z] [VERBOSE] Perform operation 'Invoke CimMethod' with following parameters, ''methodName' = SendMetaConfigurationApply,'className' = MSFT_DSCLocalConfigurationManager,'namespaceName' = root/Microsoft/Windows/DesiredStateConfiguration'. [2021-02-18 10:38:38Z] [VERBOSE] An LCM method call arrived from computer DC2-SHM-TESTB with user sid S-1-5-18. [2021-02-18 10:38:38Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ Start Set ] [2021-02-18 10:38:38Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ Start Resource ] [MSFT_DSCMetaConfiguration] [2021-02-18 10:38:38Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ Start Set ] [MSFT_DSCMetaConfiguration] [2021-02-18 10:38:38Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ End Set ] [MSFT_DSCMetaConfiguration] in 0.0160 seconds. [2021-02-18 10:38:38Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ End Resource ] [MSFT_DSCMetaConfiguration] [2021-02-18 10:38:38Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ End Set ] [2021-02-18 10:38:38Z] [VERBOSE] [DC2-SHM-TESTB]: LCM: [ End Set ] in 0.0620 seconds. [2021-02-18 10:38:38Z] [VERBOSE] Operation 'Invoke CimMethod' complete. [2021-02-18 10:38:38Z] [VERBOSE] Set-DscLocalConfigurationManager finished in 0.183 seconds. 2021-02-18 10:38:56 [ INFO]: CreateADBDC: ComponentStatus/DscExtensionLog/succeeded [2021-02-18 10:37:58Z] Extension request for sequence number 0 attempting to create lock.0 mutex [2021-02-18 10:37:58Z] Attempting to grab mutex DscExtensionHandler_Lock for sequence number 0 [2021-02-18 10:37:58Z] Acquired lock for extension instance for sequence number 0 [2021-02-18 10:37:58Z] Attempting to acquire extension lock [2021-02-18 10:37:58Z] Attempting to grab mutex DscExtensionHandler_Lock [2021-02-18 10:37:58Z] Acquired lock for extension [2021-02-18 10:37:58Z] lock does not exist: begin processing [2021-02-18 10:37:58Z] Starting DSC Extension ... [2021-02-18 10:37:58Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:37:58Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:37:58Z] Reading handler settings from C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\RuntimeSettings\0.settings [2021-02-18 10:37:58Z] Found protected settings on Azure VM. Decrypting. [2021-02-18 10:38:02Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:38:02Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:38:02Z] Updating execution status (HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status) [2021-02-18 10:38:02Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:38:02Z] Will continue the existing configuration. Executing Start-DscConfiguration with -UseExisting option ... [2021-02-18 10:38:03Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:38:36Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:38:37Z] LCM state is Idle [2021-02-18 10:38:37Z] DSC configuration completed. [2021-02-18 10:38:37Z] Resetting metaconfiguration... [2021-02-18 10:38:37Z] Restoring C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\DSCWork\CreateADBDC.0\CreateADBDC\localhost.meta.mof.bk... [2021-02-18 10:38:37Z] Executing Set-DscLocalConfigurationManager... [2021-02-18 10:38:37Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:38:39Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:38:39Z] Get-DscLocalConfigurationManager: ActionAfterReboot : ContinueConfiguration AgentId : 7B9095BB-71D4-11EB-B1A9-0022484145C6 AllowModuleOverWrite : False CertificateID : 2C493A2A96AFCBCF479F7734CB9C63757198D9DD ConfigurationDownloadManagers : {} ConfigurationID : ConfigurationMode : ApplyOnly ConfigurationModeFrequencyMins : 15 Credential : DebugMode : {NONE} DownloadManagerCustomData : DownloadManagerName : LCMCompatibleVersions : {1.0, 2.0} LCMState : Idle LCMStateDetail : LCMVersion : 2.0 StatusRetentionTimeInDays : 10 SignatureValidationPolicy : NONE SignatureValidations : {} MaximumDownloadSizeMB : 500 PartialConfigurations : RebootNodeIfNeeded : True RefreshFrequencyMins : 30 RefreshMode : PUSH ReportManagers : {} ResourceModuleManagers : {} PSComputerName : [2021-02-18 10:38:39Z] Settings handler status to 'success' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) 2021-02-18 10:38:56 [ INFO]: CreateADBDC: ComponentStatus/Metadata/succeeded VMUUId=59920B6A-C956-4D25-A5A3-53E955101CD9;AgentId=7B9095BB-71D4-11EB-B1A9-0022484145C6; 2021-02-18 10:38:56 [ INFO]: CreateADForest: ProvisioningState/succeeded DSC configuration was applied successfully. 2021-02-18 10:38:56 [ INFO]: CreateADForest: ComponentStatus/DscConfigurationLog/succeeded [2021-02-18 10:30:18Z] [WARNING] [DC1-SHM-TESTB]: [[xPendingReboot]RebootAfterPromotion] Unable to query CCM_ClientUtilities: Invalid namespace [2021-02-18 10:30:18Z] [VERBOSE] [DC1-SHM-TESTB]: [[xPendingReboot]RebootAfterPromotion] No pending reboots found. [2021-02-18 10:30:18Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ End Test ] [[xPendingReboot]RebootAfterPromotion] in 0.3750 seconds. [2021-02-18 10:30:18Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ Skip Set ] [[xPendingReboot]RebootAfterPromotion] [2021-02-18 10:30:18Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ End Resource ] [[xPendingReboot]RebootAfterPromotion] [2021-02-18 10:30:18Z] [VERBOSE] [DC1-SHM-TESTB]: [] Consistency check completed. [2021-02-18 10:30:18Z] [VERBOSE] Operation 'Invoke CimMethod' complete. [2021-02-18 10:30:18Z] [VERBOSE] Time taken for configuration job to complete is 338.069 seconds [2021-02-18 10:30:21Z] [VERBOSE] Performing the operation "Start-DscConfiguration: SendMetaConfigurationApply" on target "MSFT_DSCLocalConfigurationManager". [2021-02-18 10:30:21Z] [VERBOSE] Perform operation 'Invoke CimMethod' with following parameters, ''methodName' = SendMetaConfigurationApply,'className' = MSFT_DSCLocalConfigurationManager,'namespaceName' = root/Microsoft/Windows/DesiredStateConfiguration'. [2021-02-18 10:30:21Z] [VERBOSE] An LCM method call arrived from computer DC1-SHM-TESTB with user sid S-1-5-18. [2021-02-18 10:30:21Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ Start Set ] [2021-02-18 10:30:21Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ Start Resource ] [MSFT_DSCMetaConfiguration] [2021-02-18 10:30:21Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ Start Set ] [MSFT_DSCMetaConfiguration] [2021-02-18 10:30:22Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ End Set ] [MSFT_DSCMetaConfiguration] in 0.0310 seconds. [2021-02-18 10:30:22Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ End Resource ] [MSFT_DSCMetaConfiguration] [2021-02-18 10:30:22Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ End Set ] [2021-02-18 10:30:22Z] [VERBOSE] [DC1-SHM-TESTB]: LCM: [ End Set ] in 0.0780 seconds. [2021-02-18 10:30:22Z] [VERBOSE] Operation 'Invoke CimMethod' complete. [2021-02-18 10:30:22Z] [VERBOSE] Set-DscLocalConfigurationManager finished in 0.223 seconds. 2021-02-18 10:38:56 [ INFO]: CreateADForest: ComponentStatus/DscExtensionLog/succeeded [2021-02-18 10:24:31Z] Extension request for sequence number 0 attempting to create lock.0 mutex [2021-02-18 10:24:31Z] Attempting to grab mutex DscExtensionHandler_Lock for sequence number 0 [2021-02-18 10:24:31Z] Acquired lock for extension instance for sequence number 0 [2021-02-18 10:24:31Z] Attempting to acquire extension lock [2021-02-18 10:24:31Z] Attempting to grab mutex DscExtensionHandler_Lock [2021-02-18 10:24:31Z] Acquired lock for extension [2021-02-18 10:24:31Z] lock does not exist: begin processing [2021-02-18 10:24:31Z] Starting DSC Extension ... [2021-02-18 10:24:31Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:24:31Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:24:31Z] Reading handler settings from C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\RuntimeSettings\0.settings [2021-02-18 10:24:31Z] Found protected settings on Azure VM. Decrypting. [2021-02-18 10:24:36Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:24:36Z] Getting handler execution status HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status ... [2021-02-18 10:24:36Z] Updating execution status (HKLM:\SOFTWARE\Microsoft\Azure\DSC\2.83.0.0\Status) [2021-02-18 10:24:36Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:24:36Z] Will continue the existing configuration. Executing Start-DscConfiguration with -UseExisting option ... [2021-02-18 10:24:37Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:30:19Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:30:20Z] LCM state is Idle [2021-02-18 10:30:20Z] DSC configuration completed. [2021-02-18 10:30:20Z] Resetting metaconfiguration... [2021-02-18 10:30:20Z] Restoring C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\DSCWork\CreateADPDC.0\CreateADPDC\localhost.meta.mof.bk... [2021-02-18 10:30:20Z] Executing Set-DscLocalConfigurationManager... [2021-02-18 10:30:20Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:30:23Z] Settings handler status to 'transitioning' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) [2021-02-18 10:30:23Z] Get-DscLocalConfigurationManager: ActionAfterReboot : ContinueConfiguration AgentId : 369DEE93-71D2-11EB-B1A9-0022484145D2 AllowModuleOverWrite : False CertificateID : 5B4AC445A337958BDEBFA9B3DA4AB93AD04EBC9B ConfigurationDownloadManagers : {} ConfigurationID : ConfigurationMode : ApplyOnly ConfigurationModeFrequencyMins : 15 Credential : DebugMode : {NONE} DownloadManagerCustomData : DownloadManagerName : LCMCompatibleVersions : {1.0, 2.0} LCMState : Idle LCMStateDetail : LCMVersion : 2.0 StatusRetentionTimeInDays : 10 SignatureValidationPolicy : NONE SignatureValidations : {} MaximumDownloadSizeMB : 500 PartialConfigurations : RebootNodeIfNeeded : True RefreshFrequencyMins : 30 RefreshMode : PUSH ReportManagers : {} ResourceModuleManagers : {} PSComputerName : [2021-02-18 10:30:23Z] Settings handler status to 'success' (C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.0.0\Status\0.status) 2021-02-18 10:38:56 [ INFO]: CreateADForest: ComponentStatus/Metadata/succeeded VMUUId=EF2C433D-85FA-4B6C-BEE0-3B4A86988D4F;AgentId=369DEE93-71D2-11EB-B1A9-0022484145D2; 2021-02-18 10:38:56 [ INFO]: bginfo: ProvisioningState/succeeded Plugin enabled (handler name: Microsoft.Compute.bginfo, extension name: , version: 2.1). 2021-02-18 10:38:56 [ INFO]: bginfo: ProvisioningState/succeeded Plugin enabled (handler name: Microsoft.Compute.bginfo, extension name: , version: 2.1). 2021-02-18 10:38:56 [SUCCESS]: [✔] Template deployment 'shm-dc-template' succeeded 2021-02-18 10:38:56 [ INFO]: Importing configuration artifacts for: DC1-SHM-TESTB... 2021-02-18 10:40:33 [ INFO]: Script output: Value[0] : Code : ComponentStatus/StdOut/succeeded Level : Info DisplayStatus : Provisioning succeeded Message : Clearing all pre-existing files and folders from '"C:\Installation"' Downloading 7 files to '"C:\Installation"'... Downloading AzureADConnect to '"C:\Installation"'... [x] Failed to download AzureADConnect Extracting zip files... Value[1] : Code : ComponentStatus/StdErr/succeeded Level : Info DisplayStatus : Provisioning succeeded Message : 'Uri'. Cannot convert value "https://"shmtestbartifactsqtxiyzd".blob.core.win dows.net/"shm-configuration-dc"/UpdateAADSyncRule.ps1"?sv=2019-07-07&sig=4gDoYw%2BSin3hWxXJH%2FOJ1jV8qWWEHSTcMWFJWtOU8i U%3D&se=2021-02-18T12%3A39%3A00Z&srt=sco&ss=bf&sp=rl"" to type "System.Uri". Error: "Invalid URI: The hostname could not be parsed." At C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.8\Downloads\script0.ps1:45 char:36 + $null = Invoke-WebRequest -Uri $blobUrl -OutFile $filePath + ~~~~~~~~ + CategoryInfo : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeWebRequestCommand Join-Path : Cannot find drive. A drive with the name '"C' does not exist. At C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.8\Downloads\script0.ps1:39 char:20 + $fileDirFull = Join-Path $remoteDir $fileDirRel + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: ("C:String) [Join-Path], DriveNotFoundException + FullyQualifiedErrorId : DriveNotFound,Microsoft.PowerShell.Commands.JoinPathCommand Test-Path : Cannot bind argument to parameter 'Path' because it is null. At C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.8\Downloads\script0.ps1:40 char:31 + if (-not (Test-Path -Path $fileDirFull)) { + ~~~~~~~~~~~~ + CategoryInfo : InvalidData: (:) [Test-Path], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.TestPathCom mand Join-Path : Cannot bind argument to parameter 'Path' because it is null. At C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.8\Downloads\script0.ps1:43 char:27 + $filePath = Join-Path $fileDirFull $fileName + ~~~~~~~~~~~~ + CategoryInfo : InvalidData: (:) [Join-Path], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.JoinPathCom mand Invoke-WebRequest : Cannot bind parameter 'Uri'. Cannot convert value "https://"shmtestbartifactsqtxiyzd".blob.core.win dows.net/"shm-configuration-dc"/user_details_template.csv""?sv=2019-07-07&sig=4gDoYw%2BSin3hWxXJH%2FOJ1jV8qWWEHSTcMWFJW tOU8iU%3D&se=2021-02-18T12%3A39%3A00Z&srt=sco&ss=bf&sp=rl"" to type "System.Uri". Error: "Invalid URI: The hostname could not be parsed." At C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.8\Downloads\script0.ps1:45 char:36 + $null = Invoke-WebRequest -Uri $blobUrl -OutFile $filePath + ~~~~~~~~ + CategoryInfo : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeWebRequestCommand Invoke-WebRequest : Cannot find drive. A drive with the name '"C' does not exist. At C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.8\Downloads\script0.ps1:49 char:1 + Invoke-WebRequest -Uri https://download.microsoft.com/download/B/0/0/ ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: ("C:String) [Invoke-WebRequest], DriveNotFoundException + FullyQualifiedErrorId : DriveNotFound,Microsoft.PowerShell.Commands.InvokeWebRequestCommand New-Item : Cannot find drive. A drive with the name '"C' does not exist. At C:\windows\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:318 char:32 + ... eatedItem = New-Item -Path $DestinationPath -ItemType Directory -Conf ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: ("C:String) [New-Item], DriveNotFoundException + FullyQualifiedErrorId : DriveNotFound,Microsoft.PowerShell.Commands.NewItemCommand Status : Succeeded Capacity : 0 Count : 0 2021-02-18 10:40:33 [FAILURE]: [x] Remote script execution has failed. Please check the output above before re-running this script. Exception: /Users/echalstrey/projects/data-safe-haven/deployment/common/Logging.psm1:42 Line | 42 | throw "$Message" | ~~~~~~~~~~~~~~~~ | Remote script execution has failed. Please check the output above before re-running this script. ```
martintoreilly commented 3 years ago

@JimMadge Please could you pick this up in James' absence? It would be good to have a new release to base the DWP deployment from. Thanks.

martintoreilly commented 3 years ago

Will this be a major (v4) or minor release (v3.2) and will it be a full release or a pre-release? If it's a full release, let's get a penetration test scheduled with Ian.

JimMadge commented 3 years ago

Combining with https://github.com/alan-turing-institute/data-safe-haven/issues/932 to verify the check list against a new deployment at this commit hash.

edwardchalstrey1 commented 3 years ago

@jemrobinson I'm picking this up - I've made a branch from this commit in case anyone pushes additional commits to master: release-candidate-944

edwardchalstrey1 commented 3 years ago

Due the error we're looking into on #948 which is preventing progress here - I'm going to start a fresh deployment using testc (in case the issue is related to teardown not tidying things nicely, although this may be unlikely - see the issue thread)

edwardchalstrey1 commented 3 years ago

SHM 'testc' deployment

Setup_SHM_DNS_Zone ```pwsh ./Setup_SHM_DNS_Zone.ps1 -shmId testc 2021-02-19 15:16:22 [ INFO]: Ensuring that resource group 'RG_SHM_DNS_TEST' exists... 2021-02-19 15:16:22 [SUCCESS]: [✔] Resource group 'RG_SHM_DNS_TEST' already exists 2021-02-19 15:16:22 [ INFO]: Ensuring that DNS Zone exists... 2021-02-19 15:16:22 [ INFO]: Ensuring the DNS zone 'testc.dsgroupdev.co.uk' exists... 2021-02-19 15:16:24 [SUCCESS]: [✔] DNS Zone 'testc.dsgroupdev.co.uk' already exists 2021-02-19 15:16:24 [ INFO]: Get NS records from the new DNS Zone... 2021-02-19 15:16:24 [ INFO]: Reading NS records '@' for DNS Zone 'testc.dsgroupdev.co.uk'... 2021-02-19 15:16:27 [ INFO]: Add NS records to the parent DNS Zone... 2021-02-19 15:16:29 [SUCCESS]: [✔] DNS record set 'testc' already exists. Will not update! ```
jemrobinson commented 3 years ago

@edwardchalstrey1 : is #948 a blocker to 76fa6427 being a viable release candidate? Is the problem only if you run the teardown script?

edwardchalstrey1 commented 3 years ago

@edwardchalstrey1 : is #948 a blocker to 76fa642 being a viable release candidate? Is the problem only if you run the teardown script?

@jemrobinson It may well be a blocker, unless the problem doesn't occur when you don't run a teardown script - hence starting fresh testc deploy - I haven't got far enough into the deploy yet to test this as I was having trouble with Setup_SHM_AAD_Domain.ps1 on Friday, but re-running that now - ah it just failed again - "Due to delays with DNS propagation, occasionally the script may exhaust the maximum number of retries without managing to verify the domain" - it's failing after 10 attempts - it looks like from the troubleshooting in the doc I should just try again later

the error is:

[WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'
JimMadge commented 3 years ago

@edwardchalstrey1 It shouldn't take days for the DNS changes to propagate. Could you check if the DNS records are what you expect? Maybe old records from a previous deployment are causing problems?

You will want to check records in the [Prod] Safe Haven Domains subscription too as this is where the NS records for sub-domains of dsgroupdev.co.uk are.

@jemrobinson @martintoreilly I feel like DNS configuration is a common problem when people are deploying an SHM. Maybe we need to emphasise that there is a manual step here?

martintoreilly commented 3 years ago

@JimMadge The effective time to update DNS can be many multiples of the TTL (time-to-live) set for the record due to various caches in the system. However, we set these pretty low, so generally DNS updates should propagate smoothly and in minutes rather than hours.

That said, there's an "It's always DNS meme", so 🤷‍♂️ .

martintoreilly commented 3 years ago

@edwardchalstrey1 From the error class (Microsoft.Online.Workflows.DomainOperationNotAllowedException) this doesn't feel like a DNS propagation problem.

martintoreilly commented 3 years ago

@edwardchalstrey1 I found a post that suggest this error is raised if the domain is already linked to another Azure Active Directory tenant. Have you by any chance deleted and re-created the testc AAD tenant? Deleting an AAD tenant takes a long time to complete.

martintoreilly commented 3 years ago

@edwardchalstrey1 What happens if you try manually adding the domain via the Azure portal?

JimMadge commented 3 years ago

@JimMadge The effective time to update DNS can be many multiples of the TTL (time-to-live) set for the record due to various caches in the system. However, we set these pretty low, so generally DNS updates should propagate smoothly and in minutes rather than hours.

It doesn't help that the time taken for updates to propagate is not predictable.

I have some (limited) experience with a simple DNS setup where the registrar DNS is pointing to your cloud providers DNS; I think this is essentially the case here. In that situation I've found once you have the NS records set up with your registrar you don't have to wait for updates you make on your cloud providers DNS to take effect. Presumably the reason is all public DNS now point to your cloud providers DNS, and any changes you make within your cloud providers DNS take effect immediately and do not need to propagate.

edwardchalstrey1 commented 3 years ago

Starting again after deleting the testc.dsgroupdev.co.uk domain (to get rid of old records):

SHM 'testc' deployment

Setup_SHM_DNS_Zone ```pwsh > ./Setup_SHM_DNS_Zone.ps1 -shmId testc 2021-02-26 11:31:40 [ INFO]: Ensuring that resource group 'RG_SHM_DNS_TEST' exists... 2021-02-26 11:31:40 [SUCCESS]: [✔] Resource group 'RG_SHM_DNS_TEST' already exists 2021-02-26 11:31:40 [ INFO]: Ensuring that DNS Zone exists... 2021-02-26 11:31:40 [ INFO]: Ensuring the DNS zone 'testc.dsgroupdev.co.uk' exists... 2021-02-26 11:31:43 [ INFO]: [ ] Creating DNS Zone 'testc.dsgroupdev.co.uk' 2021-02-26 11:31:48 [SUCCESS]: [✔] Created DNS Zone 'testc.dsgroupdev.co.uk' 2021-02-26 11:31:48 [ INFO]: Get NS records from the new DNS Zone... 2021-02-26 11:31:48 [ INFO]: Reading NS records '@' for DNS Zone 'testc.dsgroupdev.co.uk'... 2021-02-26 11:31:51 [ INFO]: Add NS records to the parent DNS Zone... 2021-02-26 11:31:54 [SUCCESS]: [✔] DNS record set 'testc' already exists. Will not update! ```
Setup_SHM_AAD_Domain ```pwsh > pwsh { ./Setup_SHM_AAD_Domain.ps1 -shmId testc -tenantId 07cf5392-87da-4c5f-8676-7c62fc1a625a } WARNING: /Users/echalstrey/.local/share/powershell/Modules/AzureAD.Standard.Preview/0.1.599.7 WARNING: AzureAD.Standard.Preview WARNING: /Users/echalstrey/.local/share/powershell/Modules/AzureAD.Standard.Preview/0.1.599.7/net471 WARNING: Loading module in coreclr folder: 'netstandard2.0' ... Connecting to Azure AD '07cf5392-87da-4c5f-8676-7c62fc1a625a'... WARNING: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code CR72BETYT to authenticate. 2021-02-26 14:16:18 [ INFO]: Adding SHM domain to AAD... 2021-02-26 14:16:19 [SUCCESS]: [✔] 'testc.dsgroupdev.co.uk' already present as custom domain on SHM AAD. 2021-02-26 14:16:19 [ INFO]: Verifying domain on SHM AAD... 2021-02-26 14:16:21 [SUCCESS]: [✔] Verification TXT record already exists in 'testc.dsgroupdev.co.uk' DNS zone. 2021-02-26 14:16:21 [ INFO]: Checking domain verification status on SHM AAD (attempt 1 of 10)... 2021-02-26 14:16:21 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:16:21 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:17:21 [ INFO]: Checking domain verification status on SHM AAD (attempt 2 of 10)... 2021-02-26 14:17:22 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:17:22 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:18:22 [ INFO]: Checking domain verification status on SHM AAD (attempt 3 of 10)... 2021-02-26 14:18:22 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:18:23 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:19:23 [ INFO]: Checking domain verification status on SHM AAD (attempt 4 of 10)... 2021-02-26 14:19:23 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:19:23 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:20:23 [ INFO]: Checking domain verification status on SHM AAD (attempt 5 of 10)... 2021-02-26 14:20:23 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:20:23 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:21:23 [ INFO]: Checking domain verification status on SHM AAD (attempt 6 of 10)... 2021-02-26 14:21:24 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:21:24 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:22:24 [ INFO]: Checking domain verification status on SHM AAD (attempt 7 of 10)... 2021-02-26 14:22:24 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:22:24 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:23:24 [ INFO]: Checking domain verification status on SHM AAD (attempt 8 of 10)... 2021-02-26 14:23:25 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:23:25 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:24:25 [ INFO]: Checking domain verification status on SHM AAD (attempt 9 of 10)... 2021-02-26 14:24:25 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:24:25 [WARNING]: Verification check failed. Retrying in 60 seconds... 2021-02-26 14:25:25 [ INFO]: Checking domain verification status on SHM AAD (attempt 10 of 10)... 2021-02-26 14:25:26 [WARNING]: Domain verification failed with the following error: 'Domain takeover failed paramName: name, paramValue: testc.dsgroupdev.co.uk, objectType: Microsoft.Online.Workflows.DomainOperationNotAllowedException'. 2021-02-26 14:25:26 [FAILURE]: [x] Failed to verify domain after 10 attempts. Please try again later. Exception: /Users/echalstrey/projects/data-safe-haven/deployment/common/Logging.psm1:42 Line | 42 | throw "$Message" | ~~~~~~~~~~~~~~~~ | Failed to verify domain after 10 attempts. Please try again later. ```

as of 11.50 (and again 14:30) on 26th Feb I'm still getting Domain verification failing at the next step, but on discussion with @JimMadge I will try again in an hour or so to give changes time to propagate

edwardchalstrey1 commented 3 years ago

@edwardchalstrey1 I found a post that suggest this error is raised if the domain is already linked to another Azure Active Directory tenant. Have you by any chance deleted and re-created the testc AAD tenant? Deleting an AAD tenant takes a long time to complete.

I think if you look at the above comment @martintoreilly that what @JimMadge and I did earlier was re-create/re-verify the testc.dsgroupdev.co.uk subdomain by deleting it and then running the Setup_SHM_DNS_Zone script again - if you look at the AAD setup terminal output in the comment above it includes [SUCCESS]: [✔] Verification TXT record already exists in 'testc.dsgroupdev.co.uk' DNS zone.

But we didn't delete the testc AAD tenant before - do you think I should do that?

edwardchalstrey1 commented 3 years ago

I have tried again today with deleting the AAD tenant and running a teardown to remove any resource groups for testc but I'm still getting the same issue with Domain verification failing, although this time get a new TXT record this time 2021-03-02 10:32:42 [SUCCESS]: [✔] Verification TXT record added to 'testc.dsgroupdev.co.uk' DNS zone. which is different from the above comment

martintoreilly commented 3 years ago

I'd move to a new SHM name. It can take months for an AAD tenant to be properly deleted without raising a service ticket with Microsoft.

edwardchalstrey1 commented 3 years ago

ah ok sure, I'll just start with a fresh one then, call it testd - sorry I haven't focussed on this today but I think #948 will block my progress later on anyway

jemrobinson commented 3 years ago

This deployment is superseded by #970. Closing.