jdm019000
commented
3 years ago Update after debugging all day yesterday.
- Yesterday we found that the ASDK baseline installer script did not add the SQL cluster name DNS A record and IP to the DNS. It also did not add the SQL cluster name computer object to the AD. The script was looking for both of those and will fail if they are not there. Once we added them, then the rerun of the script progressed.
- Last night the script ran from 5pm to around 10p and is now in a hung state. It did not exit and return to c prompt but seems to be hung. It is having issue with the storage accounts. It seems to be hung for around 12 hours now.
a. The failover cluster manager events shows a recurring error.
Cluster resource "Virtual Machine AzS-ACS01" of type "Virual Machine" in clustered role failed.
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on the this node or move the group to another node and then restart it.
Check the resouce and group state using failover cluster manager or the Get-ClusterResource Windows PowerShell cmdlet. b. The last stage on the PS console shows the below. The create logs statement C:\MASLogs\StorageAccounts_CreateStorageAccounts_20201105-022653.log on the server AzS-Xrp01.azurestack.local is 0 bytes empty. So not able to find anything. I am not sure what to do at this stage.
STARTING: Task Cloud\Fabric\StorageAccounts - Configure VERBOSE: 1> Interface: Path to module: C:\CloudDeployment\Classes\StorageAccounts\StorageAccounts.psm1 - 11/5/2020 2:25:53 AM VERBOSE: 1> Interface: Running interface Configure (Classes\StorageAccounts\StorageAccounts.psm1, StorageAccounts:Configure) - 11/5/2020 2:25:53 AM VERBOSE: 1> [PSTask Concurrency] [63>StorageAccounts:Configure] Entering throttling instance, enqueue for waiting. Priority '0', Weight '10'. - 11/5/2020 2:25:53 AM VERBOSE: 1> [PSTask Concurrency] [63>StorageAccounts:Configure] Validating availability to start task. - 11/5/2020 2:25:53 AM VERBOSE: 1> [PSTask Concurrency] [63>StorageAccounts:Configure] Task available to start. - 11/5/2020 2:25:53 AM VERBOSE: 1> [PSTask Concurrency] [63>StorageAccounts:Configure] Parallel task count increased by '10' to '10'. - 11/5/2020 2:25:53 AM VERBOSE: 1> [PSTask Concurrency] [63>StorageAccounts:Configure] Throttling instance dequeued from waiting. Priority '0', Weight '10'. - 11/5/2020 2:25:53 AM VERBOSE: 1> [PSTask Concurrency] [63>StorageAccounts:Configure] TotalVirtualMemoryMB: [131071.55078125], FreeVirtualMemoryMB: [59127.4296875],FreePhysicalMemoryMB: [59655.9296875]. - 11/5/2020 2:25:53 AM WARNING: 1> [StorageAccounts:Configure] The names of some imported commands from the module 'StorageAccounts' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. - 11/5/2020 2:26:21 AM VERBOSE: 1> [StorageAccounts:Configure] Initializing remote powershell session on AzS-Xrp01.azurestack.local with common functions. - 11/5/2020 2:26:51 AM VERBOSE: 1> [StorageAccounts:Configure] Transcript started, output file is C:\MASLogs\StorageAccounts_CreateStorageAccounts_20201105-022653.log - 11/5/2020 2:26:54 AM VERBOSE: 1> [StorageAccounts:Configure] Registering the admin subscriptions [abaf2eb8-71b9-4d50-9c30-0c32668f6703 6d8d45aa-3e27-4914-96bb-23706ad8a66b] with the 'Microsoft.Storage' namespace - 11/5/2020 2:26:54 AM
rtibi
Any Suggestions here. Trying to deploy ASDK 2005. All of the roles seem to be up. Not sure what direction I should go.
`2020-11-04 14:22:55 Verbose Step:Skipping task 'Deployment-Phase3-DeployADFS' of role 'Cloud' because it was skipped or run successfully previously. 2020-11-04 14:22:55 Verbose 5> Creating action host for action Deployment-Phase3-DeploySQL. 2020-11-04 14:22:55 Verbose 5> Found the role 'Cloud' in the current configuration. 2020-11-04 14:22:55 Verbose 5> Using the existing expanded action definition for action Deployment-Phase3-DeploySQL. 2020-11-04 14:22:55 Verbose 5> Creating action host for action Deployment-Phase3-DeploySQL. 2020-11-04 14:22:55 Verbose 5> Action host for action Deployment-Phase3-DeploySQL has been created. 2020-11-04 14:22:55 Verbose 5> Update the actionTask definition with the new nested actionHost definition for Cloud - Deployment-Phase3-DeploySQL. 2020-11-04 14:22:55 Verbose 5> Task: Running action type 'Deployment-Phase3-DeploySQL' of role 'Cloud'. 2020-11-04 14:22:55 Verbose 5> Action: Running action Deployment-Phase3-DeploySQL. 2020-11-04 14:22:55 Verbose 5> Action: Running action plan 'Deployment-Phase3-DeploySQL'. 2020-11-04 14:22:55 Verbose 5> The following steps are currently ready: 60.62.101. 2020-11-04 14:22:55 Verbose 5> Processing step: 60.62.101. 2020-11-04 14:22:55 Verbose 5> Executing step: 60.62.101. 2020-11-04 14:22:55 Verbose 5> Checking if any of the in progress steps are complete. The following steps are currently in progress: '60.62.101'. 2020-11-04 14:22:55 Verbose 5> The following steps are currently ready: . 2020-11-04 14:22:55 Verbose 5> No steps are ready for execution. Waiting for any of the currently-running steps to complete. The following steps are currently in progress: '60.62.101'. 2020-11-04 14:22:55 Verbose 5> Step: Running step 60.62.101 - (FBI) Setup SQL VMs 2020-11-04 14:22:55 Verbose 5> Task: Running interface 'Configure' of role 'Cloud\Fabric\SQL'. Attempt #1 of 3 2020-11-04 14:22:55 Verbose 5> Loading CustomerConfiguration for interface 'Configure' of role 'Cloud\Fabric\SQL'. 2020-11-04 14:22:55 Verbose 5> Task: Interface 'Configure' is getting created. 2020-11-04 14:22:55 Verbose 5> Interface: Path to module: C:\CloudDeployment\Classes\SQL\SQL.psm1 2020-11-04 14:22:55 Verbose 5> Interface: Running interface Configure (Classes\SQL\SQL.psm1, SQL:Configure) 2020-11-04 14:22:55 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] Entering throttling instance, enqueue for waiting. Priority '0', Weight '10'. 2020-11-04 14:22:55 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] Validating availability to start task. 2020-11-04 14:22:55 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] Task available to start. 2020-11-04 14:22:55 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] Parallel task count increased by '10' to '10'. 2020-11-04 14:22:55 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] Throttling instance dequeued from waiting. Priority '0', Weight '10'. 2020-11-04 14:22:55 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] TotalVirtualMemoryMB: [131071.55078125], FreeVirtualMemoryMB: [58925.70703125], FreePhysicalMemoryMB: [59237.52734375]. 2020-11-04 14:23:08 Warning 5> [SQL:Configure] The names of some imported commands from the module 'RoleHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:10 Warning 5> [SQL:Configure] The names of some imported commands from the module 'NetworkHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:10 Warning 5> [SQL:Configure] The names of some imported commands from the module 'RoleHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:15 Warning 5> [SQL:Configure] The names of some imported commands from the module 'RoleHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:16 Warning 5> [SQL:Configure] The names of some imported commands from the module 'RoleHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:16 Warning 5> [SQL:Configure] The names of some imported commands from the module 'RoleHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:17 Warning 5> [SQL:Configure] The names of some imported commands from the module 'RoleHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:22 Warning 5> [SQL:Configure] The names of some imported commands from the module 'NetworkHelpers' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:22 Warning 5> [SQL:Configure] The names of some imported commands from the module 'SQL' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:23 Warning 5> [SQL:Configure] The names of some imported commands from the module 'SQL' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2020-11-04 14:23:23 Verbose 5> [SQL:Configure] Execution start of New-Sql. 2020-11-04 14:23:30 Verbose 5> [SQL:Configure] Prepare nodes for clustering: 'AzS-Sql01.azurestack.local' 2020-11-04 14:23:33 Verbose 5> [SQL:Configure] Enabling client/diagnostic event log on AzS-Sql01.azurestack.local 2020-11-04 14:23:40 Verbose 5> [SQL:Configure] Clean up the AD computer object AzS-SqlCl and its DNS record 2020-11-04 14:23:40 Verbose 5> [SQL:Configure] Removing DNS resource record for name 'AzS-SqlCl' with IP 'True' under zone name 'azurestack.local' on the domain controller 'AzS-DC01'. 2020-11-04 14:23:45 Verbose 5> [SQL:Configure] Record for 'True' (A) not present. 2020-11-04 14:23:45 Verbose 5> [SQL:Configure] Removing DNS resource record for name 'AzS-SqlCl' with IP 'True' under zone name 'azurestack.local' on the domain controller 'AzS-DC01'. 2020-11-04 14:23:45 Verbose 5> [SQL:Configure] Record for 'True' (AAAA) not present. 2020-11-04 14:23:45 Verbose 5> [SQL:Configure] Creating the cluster 'AzS-SqlCl'. 2020-11-04 14:24:08 Verbose 5> [SQL:Configure] Collecting client/diagnostic event log on AzS-Sql01.azurestack.local 2020-11-04 14:24:32 Verbose 5> [SQL:Configure] ECE PowerShellHost Script End 2020-11-04 14:24:32 Verbose 5> Interface: PowerShell job terminal state is Failed for interface 'Configure'. 2020-11-04 14:24:33 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] Exiting throttling instance. Priority '0', Weight '10'. 2020-11-04 14:24:33 Verbose 5> [PSTask Concurrency] [113>SQL:Configure] Parallel task count decreased by '10' to '0'. 2020-11-04 14:24:33 Warning 5> Task: Invocation of interface 'Configure' of role 'Cloud\Fabric\SQL' failed:
Type 'Configure' of Role 'SQL' raised an exception:
There was an error adding node 'AzS-Sql01.azurestack.local' to the cluster The computer 'AzS-Sql01.azurestack.local' is joined to a cluster. Command Arguments
Add-GuestCluster {Parameters=CloudEngine.Configurations.EceInterfaceParameters, RunAsUserID=Fabric}
New-Sql {Parameters=CloudEngine.Configurations.EceInterfaceParameters, ErrorAction=Stop, Verbose=True}
{}