Azure / azure-powershell

Microsoft Azure PowerShell
Other
4.26k stars 3.86k forks source link

Restore-AzRecoveryServicesBackupItem is inconsistent in whether it succeeds or fails #15959

Closed mwsutherland closed 3 years ago

mwsutherland commented 3 years ago

Description

Sometimes Restore-AzRecoveryServicesBackupItem fails for no obvious reason. I've had the same database sometimes fail and sometimes succeed. My intended use case has me trying to restore two databases. The first fails and the second succeeds. If I swap the order, the new first fails and the new second succeeds. In testing with a single database, I find that it will consistently either fail or succeed for quite a while. I started working on this ticket on a Friday. Everything was failing as I put together my demonstration script. The work day was over and when I ran it on Monday to gather output, it worked. The good news is that, after waiting for a few hours, it failed again, so now I have output for both cases. Comparing the output, the only thing I notice different before the error, other than time stamps and correlation IDs, is that another log backup had happened between the two attempts, so they aren't actually targeting the same point-in-time. I don't know why that should matter unless the LatestRecoveryPoint field of the backup I'm trying to restore isn't always valid point-in-time to use. That doesn't seem to explain my scenario where I swap the order of restores, but I'll investigate this angle further and add any new findings in the comments.

Steps to reproduce

Connect-AzAccount
Set-AzContext "Microsoft Azure Enterprise"
$Vault = Get-AzRecoveryServicesVault -ResourceGroupName "New-LRS-Recovery-Vault" -Name "New-LRS-Recovery-Vault"
Set-AzRecoveryServicesVaultContext -Vault $Vault
$BackupItem = Get-AzRecoveryServicesBackupItem -BackupManagementType "AzureWorkload" -WorkloadType "MSSQL" -Name "SQLDataBase;MSSQLSERVER;GSDWCON"
$TargetInstance = Get-AzRecoveryServicesBackupProtectableItem -WorkloadType "MSSQL" -ItemType "SQLInstance" -ServerName "dwqa.gassouth.com" -Name "sqlinstance;mssqlserver"
$TargetContainer = Get-AzRecoveryServicesBackupContainer -ContainerType "AzureVMAppContainer" -FriendlyName "DWQA"
$Config = Get-AzRecoveryServicesBackupWorkloadRecoveryConfig -Item $BackupItem -PointInTime $BackupItem.LatestRecoveryPoint -TargetItem $TargetInstance -TargetContainer $TargetContainer -AlternateWorkloadRestore
$Config.RestoredDBName = "GSDWCON"
$Config.OverwriteWLIfpresent = "Yes"
$Config.targetPhysicalPath[0].TargetPath = "E:\SQLData\GSDWCON\GSDWCON.mdf"
$Config.targetPhysicalPath[1].TargetPath = "E:\SQLData\GSDWCON\GSDWCON_log.ldf"
# Display contents of parameter
$Config | ConvertTo-Json
Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config

Environment data

Name                           Value
----                           -----
PSVersion                      5.1.19041.1237
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.1237
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Module versions

Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     2.5.3      Az.Accounts                         {Disable-AzDataCollection, Disable-AzContextAutosave, Enable-AzDataCollection, Enable-AzContextAutosave...}
Script     1.1.1      Az.Advisor                          {Get-AzAdvisorRecommendation, Enable-AzAdvisorRecommendation, Disable-AzAdvisorRecommendation, Get-AzAdvisorConfigura...
Script     2.4.0      Az.Aks                              {Get-AzAksCluster, New-AzAksCluster, Remove-AzAksCluster, Import-AzAksCredential...}
Script     1.1.4      Az.AnalysisServices                 {Resume-AzAnalysisServicesServer, Suspend-AzAnalysisServicesServer, Get-AzAnalysisServicesServer, Remove-AzAnalysisSe...
Script     2.3.0      Az.ApiManagement                    {Add-AzApiManagementApiToGateway, Add-AzApiManagementApiToProduct, Add-AzApiManagementProductToGroup, Add-AzApiManage...
Script     1.0.0      Az.AppConfiguration                 {Get-AzAppConfigurationStore, Get-AzAppConfigurationStoreKey, New-AzAppConfigurationStore, New-AzAppConfigurationStor...
Script     1.2.0      Az.ApplicationInsights              {Get-AzApplicationInsights, New-AzApplicationInsights, Remove-AzApplicationInsights, Update-AzApplicationInsights...}
Script     1.7.1      Az.Automation                       {Get-AzAutomationHybridWorkerGroup, Remove-AzAutomationHybridWorkerGroup, Get-AzAutomationJobOutputRecord, Import-AzA...
Script     3.1.0      Az.Batch                            {Remove-AzBatchAccount, Get-AzBatchAccount, Get-AzBatchAccountKey, New-AzBatchAccount...}
Script     2.0.0      Az.Billing                          {Get-AzBillingInvoice, Get-AzBillingPeriod, Get-AzEnrollmentAccount, Get-AzConsumptionBudget...}
Script     1.8.0      Az.Cdn                              {Get-AzCdnProfile, Get-AzCdnProfileSsoUrl, New-AzCdnProfile, Remove-AzCdnProfile...}
Script     1.9.0      Az.CognitiveServices                {Get-AzCognitiveServicesAccount, Get-AzCognitiveServicesAccountKey, Get-AzCognitiveServicesAccountSku, Get-AzCognitiv...
Script     4.17.0     Az.Compute                          {Remove-AzAvailabilitySet, Get-AzAvailabilitySet, New-AzAvailabilitySet, Update-AzAvailabilitySet...}
Script     2.1.0      Az.ContainerInstance                {Add-AzContainerInstanceOutput, Get-AzContainerGroup, Get-AzContainerInstanceCachedImage, Get-AzContainerInstanceCapa...
Script     2.2.3      Az.ContainerRegistry                {New-AzContainerRegistry, Get-AzContainerRegistry, Update-AzContainerRegistry, Remove-AzContainerRegistry...}
Script     1.3.1      Az.CosmosDB                         {Get-AzCosmosDBSqlContainer, Get-AzCosmosDBSqlContainerThroughput, Get-AzCosmosDBSqlDatabase, Get-AzCosmosDBSqlDataba...
Script     1.1.0      Az.DataBoxEdge                      {Get-AzDataBoxEdgeJob, Get-AzDataBoxEdgeDevice, Invoke-AzDataBoxEdgeDevice, New-AzDataBoxEdgeDevice...}
Script     1.1.0      Az.Databricks                       {Get-AzDatabricksVNetPeering, Get-AzDatabricksWorkspace, New-AzDatabricksVNetPeering, New-AzDatabricksWorkspace...}
Script     1.14.0     Az.DataFactory                      {Set-AzDataFactoryV2, Update-AzDataFactoryV2, Get-AzDataFactoryV2, Remove-AzDataFactoryV2...}
Script     1.0.2      Az.DataLakeAnalytics                {Get-AzDataLakeAnalyticsDataSource, New-AzDataLakeAnalyticsCatalogCredential, Remove-AzDataLakeAnalyticsCatalogCreden...
Script     1.3.0      Az.DataLakeStore                    {Get-AzDataLakeStoreTrustedIdProvider, Remove-AzDataLakeStoreTrustedIdProvider, Remove-AzDataLakeStoreFirewallRule, S...
Script     1.0.0      Az.DataShare                        {New-AzDataShareAccount, Get-AzDataShareAccount, Remove-AzDataShareAccount, New-AzDataShare...}
Script     1.1.0      Az.DeploymentManager                {Get-AzDeploymentManagerArtifactSource, New-AzDeploymentManagerArtifactSource, Set-AzDeploymentManagerArtifactSource,...
Script     3.0.0      Az.DesktopVirtualization            {Disconnect-AzWvdUserSession, Expand-AzWvdMsixImage, Get-AzWvdApplication, Get-AzWvdApplicationGroup...}
Script     1.0.2      Az.DevTestLabs                      {Get-AzDtlAllowedVMSizesPolicy, Get-AzDtlAutoShutdownPolicy, Get-AzDtlAutoStartPolicy, Get-AzDtlVMsPerLabPolicy...}
Script     1.1.2      Az.Dns                              {Get-AzDnsRecordSet, New-AzDnsRecordConfig, Remove-AzDnsRecordSet, Set-AzDnsRecordSet...}
Script     1.3.0      Az.EventGrid                        {New-AzEventGridTopic, Get-AzEventGridTopic, Set-AzEventGridTopic, New-AzEventGridTopicKey...}
Script     1.8.0      Az.EventHub                         {New-AzEventHubNamespace, Get-AzEventHubNamespace, Set-AzEventHubNamespace, Remove-AzEventHubNamespace...}
Script     1.8.0      Az.FrontDoor                        {New-AzFrontDoor, Get-AzFrontDoor, Set-AzFrontDoor, Remove-AzFrontDoor...}
Script     3.1.0      Az.Functions                        {Get-AzFunctionApp, Get-AzFunctionAppAvailableLocation, Get-AzFunctionAppPlan, Get-AzFunctionAppSetting...}
Script     4.3.0      Az.HDInsight                        {Get-AzHDInsightJob, New-AzHDInsightSqoopJobDefinition, Wait-AzHDInsightJob, New-AzHDInsightStreamingMapReduceJobDefi...
Script     1.3.1      Az.HealthcareApis                   {New-AzHealthcareApisService, Remove-AzHealthcareApisService, Set-AzHealthcareApisService, Get-AzHealthcareApisService}
Script     2.7.3      Az.IotHub                           {Add-AzIotHubKey, Get-AzIotHubEventHubConsumerGroup, Get-AzIotHubConnectionString, Get-AzIotHubJob...}
Script     3.5.0      Az.KeyVault                         {Add-AzKeyVaultCertificate, Update-AzKeyVaultCertificate, Stop-AzKeyVaultCertificateOperation, Get-AzKeyVaultCertific...
Script     2.0.0      Az.Kusto                            {Add-AzKustoClusterLanguageExtension, Add-AzKustoDatabasePrincipal, Get-AzKustoAttachedDatabaseConfiguration, Get-AzK...
Script     1.5.0      Az.LogicApp                         {Get-AzIntegrationAccountAgreement, Get-AzIntegrationAccountAssembly, Get-AzIntegrationAccountBatchConfiguration, Get...
Script     1.1.3      Az.MachineLearning                  {Move-AzMlCommitmentAssociation, Get-AzMlCommitmentAssociation, Get-AzMlCommitmentPlanUsageHistory, Remove-AzMlCommit...
Script     1.1.1      Az.Maintenance                      {Get-AzApplyUpdate, Get-AzConfigurationAssignment, Get-AzMaintenanceConfiguration, Get-AzMaintenanceUpdate...}
Script     2.0.0      Az.ManagedServices                  {Get-AzManagedServicesAssignment, New-AzManagedServicesAssignment, Remove-AzManagedServicesAssignment, Get-AzManagedS...
Script     1.0.2      Az.MarketplaceOrdering              {Get-AzMarketplaceTerms, Set-AzMarketplaceTerms}
Script     1.1.1      Az.Media                            {Sync-AzMediaServiceStorageKey, Set-AzMediaServiceKey, Get-AzMediaServiceKey, Get-AzMediaServiceNameAvailability...}
Script     1.1.1      Az.Migrate                          {Get-AzMigrateDiscoveredServer, Get-AzMigrateJob, Get-AzMigrateProject, Get-AzMigrateReplicationEligibilityResult...}
Script     2.7.0      Az.Monitor                          {Get-AzMetricDefinition, Get-AzMetric, Remove-AzLogProfile, Get-AzLogProfile...}
Script     0.7.0      Az.NetAppFiles                      {Get-AzNetAppFilesAccount, New-AzNetAppFilesAccount, Remove-AzNetAppFilesAccount, Set-AzNetAppFilesAccount...}
Script     4.11.0     Az.Network                          {Add-AzApplicationGatewayAuthenticationCertificate, Get-AzApplicationGatewayAuthenticationCertificate, New-AzApplicat...
Script     1.1.1      Az.NotificationHubs                 {Get-AzNotificationHub, Get-AzNotificationHubAuthorizationRule, Get-AzNotificationHubListKey, Get-AzNotificationHubPN...
Script     2.3.0      Az.OperationalInsights              {New-AzOperationalInsightsAzureActivityLogDataSource, New-AzOperationalInsightsCustomLogDataSource, Disable-AzOperati...
Script     1.4.1      Az.PolicyInsights                   {Get-AzPolicyEvent, Get-AzPolicyState, Get-AzPolicyStateSummary, Get-AzPolicyRemediation...}
Script     1.1.2      Az.PowerBIEmbedded                  {Remove-AzPowerBIWorkspaceCollection, Get-AzPowerBIWorkspaceCollection, Get-AzPowerBIWorkspaceCollectionAccessKey, Ge...
Script     1.0.3      Az.PrivateDns                       {Get-AzPrivateDnsZone, Remove-AzPrivateDnsZone, Set-AzPrivateDnsZone, New-AzPrivateDnsZone...}
Script     4.6.0      Az.RecoveryServices                 {Get-AzRecoveryServicesBackupProperty, Get-AzRecoveryServicesVault, Get-AzRecoveryServicesVaultSettingsFile, New-AzRe...
Script     1.5.0      Az.RedisCache                       {Remove-AzRedisCachePatchSchedule, New-AzRedisCacheScheduleEntry, Get-AzRedisCachePatchSchedule, New-AzRedisCachePatc...
Script     1.0.0      Az.RedisEnterpriseCache             {Export-AzRedisEnterpriseCache, Get-AzRedisEnterpriseCache, Get-AzRedisEnterpriseCacheDatabase, Get-AzRedisEnterprise...
Script     1.0.3      Az.Relay                            {New-AzRelayNamespace, Get-AzRelayNamespace, Set-AzRelayNamespace, Remove-AzRelayNamespace...}
Script     0.9.0      Az.Reservations                     {Get-AzReservationOrder, Get-AzReservation, Get-AzReservationHistory, Update-AzReservation...}
Script     1.0.0      Az.ResourceMover                    {Add-AzResourceMoverMoveResource, Get-AzResourceMoverMoveCollection, Get-AzResourceMoverMoveResource, Get-AzResourceM...
Script     4.3.1      Az.Resources                        {Get-AzProviderOperation, Remove-AzRoleAssignment, Get-AzRoleAssignment, New-AzRoleAssignment...}
Script     1.0.0      Az.Security                         {Get-AzSecurityAlert, Set-AzSecurityAlert, Get-AzSecurityAutoProvisioningSetting, Set-AzSecurityAutoProvisioningSetti...
Script     1.1.0      Az.SecurityInsights                 {Get-AzSentinelAlertRuleAction, New-AzSentinelAlertRuleAction, Remove-AzSentinelAlertRuleAction, Update-AzSentinelAle...
Script     1.5.0      Az.ServiceBus                       {New-AzServiceBusNamespace, Get-AzServiceBusNamespace, Set-AzServiceBusNamespace, Remove-AzServiceBusNamespace...}
Script     3.0.1      Az.ServiceFabric                    {Add-AzServiceFabricClientCertificate, Add-AzServiceFabricNode, Add-AzServiceFabricNodeType, Get-AzServiceFabricClust...
Script     1.3.0      Az.SignalR                          {New-AzSignalR, Get-AzSignalR, Get-AzSignalRKey, New-AzSignalRKey...}
Script     3.5.0      Az.Sql                              {Get-AzSqlDatabaseTransparentDataEncryption, Get-AzSqlDatabaseTransparentDataEncryptionActivity, Set-AzSqlDatabaseTra...
Script     1.1.0      Az.SqlVirtualMachine                {New-AzSqlVM, Get-AzSqlVM, Update-AzSqlVM, Remove-AzSqlVM...}
Script     3.11.0     Az.Storage                          {Get-AzStorageAccount, Get-AzStorageAccountKey, New-AzStorageAccount, New-AzStorageAccountKey...}
Script     1.6.0      Az.StorageSync                      {Invoke-AzStorageSyncCompatibilityCheck, New-AzStorageSyncService, Get-AzStorageSyncService, Set-AzStorageSyncService...
Script     2.0.0      Az.StreamAnalytics                  {Get-AzStreamAnalyticsCluster, Get-AzStreamAnalyticsClusterStreamingJob, Get-AzStreamAnalyticsDefaultFunctionDefiniti...
Script     1.0.0      Az.Support                          {Get-AzSupportService, Get-AzSupportProblemClassification, Get-AzSupportTicket, Get-AzSupportTicketCommunication...}
Script     1.0.4      Az.TrafficManager                   {Add-AzTrafficManagerCustomHeaderToEndpoint, Remove-AzTrafficManagerCustomHeaderFromEndpoint, Add-AzTrafficManagerCus...
Script     2.8.2      Az.Websites                         {Get-AzAppServicePlan, Set-AzAppServicePlan, New-AzAppServicePlan, Remove-AzAppServicePlan...}
Script     1.1.20     dbatools                            {Select-DbaObject, Set-DbatoolsConfig, Add-DbaAgDatabase, Add-DbaAgListener...}
Script     1.0.1      Microsoft.PowerShell.Operation.V... {Get-OperationValidation, Invoke-OperationValidation}
Script     5.3.1      oh-my-posh                          {Get-PoshThemes, Set-PoshPrompt, Get-PoshInfoForV2Users, Show-Colors...}
Script     1.4.7      PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...}
Binary     1.0.0.1    PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...}
Script     3.4.0      Pester                              {Describe, Context, It, Should...}
Script     1.0.0      posh-git                            {Add-PoshGitToProfile, Expand-GitCommand, Format-GitBranchName, Get-GitBranchStatusColor...}
Script     2.2.5      PowerShellGet                       {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability...}
Script     1.0.0.1    PowerShellGet                       {Install-Module, Find-Module, Save-Module, Update-Module...}
Script     2.0.0      PSReadline                          {Get-PSReadLineKeyHandler, Set-PSReadLineKeyHandler, Remove-PSReadLineKeyHandler, Get-PSReadLineOption...}
Script     21.1.18256 SqlServer                           {Add-RoleMember, Add-SqlAvailabilityDatabase, Add-SqlAvailabilityGroupListenerStaticIp, Add-SqlAzureAuthenticationCon...

    Directory: C:\Windows\system32\WindowsPowerShell\v1.0\Modules

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Manifest   1.0.0.0    ActiveDirectory                     {Add-ADCentralAccessPolicyMember, Add-ADComputerServiceAccount, Add-ADDomainControllerPasswordReplicationPolicy, Add-...
Manifest   1.0.0.0    AppBackgroundTask                   {Disable-AppBackgroundTaskDiagnosticLog, Enable-AppBackgroundTaskDiagnosticLog, Set-AppBackgroundTaskResourcePolicy, ...
Manifest   2.0.0.0    AppLocker                           {Get-AppLockerFileInformation, Get-AppLockerPolicy, New-AppLockerPolicy, Set-AppLockerPolicy...}
Manifest   1.0.0.0    AppvClient                          {Add-AppvClientConnectionGroup, Add-AppvClientPackage, Add-AppvPublishingServer, Disable-Appv...}
Manifest   2.0.1.0    Appx                                {Add-AppxPackage, Get-AppxPackage, Get-AppxPackageManifest, Remove-AppxPackage...}
Script     1.0.0.0    AssignedAccess                      {Clear-AssignedAccess, Get-AssignedAccess, Set-AssignedAccess}
Manifest   1.0        BestPractices                       {Get-BpaModel, Get-BpaResult, Invoke-BpaModel, Set-BpaResult}
Manifest   1.0.0.0    BitLocker                           {Unlock-BitLocker, Suspend-BitLocker, Resume-BitLocker, Remove-BitLockerKeyProtector...}
Manifest   2.0.0.0    BitsTransfer                        {Add-BitsFile, Complete-BitsTransfer, Get-BitsTransfer, Remove-BitsTransfer...}
Manifest   1.0.0.0    BranchCache                         {Add-BCDataCacheExtension, Clear-BCCache, Disable-BC, Disable-BCDowngrading...}
Manifest   1.0.0.0    CimCmdlets                          {Get-CimAssociatedInstance, Get-CimClass, Get-CimInstance, Get-CimSession...}
Binary     2.0.0.0    ClusterAwareUpdating                {Get-CauPlugin, Register-CauPlugin, Unregister-CauPlugin, Invoke-CauScan...}
Manifest   1.0        ConfigCI                            {Get-SystemDriver, New-CIPolicyRule, New-CIPolicy, Get-CIPolicy...}
Manifest   1.0        ConfigDefender                      {Get-MpPreference, Set-MpPreference, Add-MpPreference, Remove-MpPreference...}
Manifest   1.0        Defender                            {Get-MpPreference, Set-MpPreference, Add-MpPreference, Remove-MpPreference...}
Manifest   1.0.2.0    DeliveryOptimization                {Delete-DeliveryOptimizationCache, Set-DeliveryOptimizationStatus, Get-DeliveryOptimizationLog, Get-DeliveryOptimizat...
Manifest   1.0        DFSN                                {Get-DfsnRoot, Remove-DfsnRoot, Set-DfsnRoot, New-DfsnRoot...}
Binary     2.0.0.0    DFSR                                {New-DfsReplicationGroup, Get-DfsReplicationGroup, Set-DfsReplicationGroup, Remove-DfsReplicationGroup...}
Manifest   2.0.0.0    DhcpServer                          {Add-DhcpServerInDC, Add-DhcpServerv4Class, Add-DhcpServerv4ExclusionRange, Add-DhcpServerv4Failover...}
Manifest   1.0.0.0    DirectAccessClientComponents        {Disable-DAManualEntryPointSelection, Enable-DAManualEntryPointSelection, Get-DAClientExperienceConfiguration, Get-DA...
Script     3.0        Dism                                {Add-AppxProvisionedPackage, Add-WindowsDriver, Add-WindowsCapability, Add-WindowsImage...}
Manifest   1.0.0.0    DnsClient                           {Resolve-DnsName, Clear-DnsClientCache, Get-DnsClient, Get-DnsClientCache...}
Manifest   2.0.0.0    DnsServer                           {Add-DnsServerConditionalForwarderZone, Add-DnsServerDirectoryPartition, Add-DnsServerForwarder, Add-DnsServerPrimary...
Manifest   1.0.0.0    EventTracingManagement              {Start-EtwTraceSession, New-EtwTraceSession, Get-EtwTraceSession, Update-EtwTraceSession...}
Manifest   2.0.0.0    FailoverClusters                    {Add-ClusterCheckpoint, Add-ClusterDisk, Add-ClusterFileServerRole, Add-ClusterGenericApplicationRole...}
Manifest   1.0.0.0    GroupPolicy                         {Backup-GPO, Block-GPInheritance, Copy-GPO, Get-GPInheritance...}
Manifest   1.0.0.0    HgsClient                           {Get-HgsAttestationBaselinePolicy, Get-HgsClientConfiguration, Test-HgsClientConfiguration, Set-HgsClientConfiguratio...
Manifest   2.0.0.0    International                       {Get-WinDefaultInputMethodOverride, Set-WinDefaultInputMethodOverride, Get-WinHomeLocation, Set-WinHomeLocation...}
Manifest   2.0.0.0    IpamServer                          {Get-IpamDhcpConfigurationEvent, Remove-IpamDhcpConfigurationEvent, Get-IpamConfigurationEvent, Remove-IpamConfigurat...
Manifest   1.0.0.0    iSCSI                               {Get-IscsiTargetPortal, New-IscsiTargetPortal, Remove-IscsiTargetPortal, Update-IscsiTargetPortal...}
Manifest   2.0.0.0    IscsiTarget                         {Add-ClusteriSCSITargetServerRole, Add-IscsiVirtualDiskTargetMapping, Checkpoint-IscsiVirtualDisk, Convert-IscsiVirtu...
Script     1.0.0.0    ISE                                 {New-IseSnippet, Import-IseSnippet, Get-IseSnippet}
Manifest   1.0.0.0    Kds                                 {Add-KdsRootKey, Get-KdsRootKey, Test-KdsRootKey, Set-KdsConfiguration...}
Manifest   1.0.1.0    Microsoft.PowerShell.Archive        {Compress-Archive, Expand-Archive}
Manifest   3.0.0.0    Microsoft.PowerShell.Diagnostics    {Get-WinEvent, Get-Counter, Import-Counter, Export-Counter...}
Manifest   3.0.0.0    Microsoft.PowerShell.Host           {Start-Transcript, Stop-Transcript}
Manifest   1.0.0.0    Microsoft.PowerShell.LocalAccounts  {Add-LocalGroupMember, Disable-LocalUser, Enable-LocalUser, Get-LocalGroup...}
Manifest   3.1.0.0    Microsoft.PowerShell.Management     {Add-Content, Clear-Content, Clear-ItemProperty, Join-Path...}
Script     1.0        Microsoft.PowerShell.ODataUtils     Export-ODataEndpointProxy
Manifest   3.0.0.0    Microsoft.PowerShell.Security       {Get-Acl, Set-Acl, Get-PfxCertificate, Get-Credential...}
Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Format-List, Format-Custom, Format-Table, Format-Wide...}
Manifest   3.0.0.0    Microsoft.WSMan.Management          {Disable-WSManCredSSP, Enable-WSManCredSSP, Get-WSManCredSSP, Set-WSManQuickConfig...}
Manifest   1.0        MMAgent                             {Disable-MMAgent, Enable-MMAgent, Set-MMAgent, Get-MMAgent...}
Manifest   1.0.0.0    MsDtc                               {New-DtcDiagnosticTransaction, Complete-DtcDiagnosticTransaction, Join-DtcDiagnosticResourceManager, Receive-DtcDiagn...
Manifest   2.0.0.0    NetAdapter                          {Disable-NetAdapter, Disable-NetAdapterBinding, Disable-NetAdapterChecksumOffload, Disable-NetAdapterEncapsulatedPack...
Manifest   1.0.0.0    NetConnection                       {Get-NetConnectionProfile, Set-NetConnectionProfile}
Manifest   1.0.0.0    NetEventPacketCapture               {New-NetEventSession, Remove-NetEventSession, Get-NetEventSession, Set-NetEventSession...}
Manifest   2.0.0.0    NetLbfo                             {Add-NetLbfoTeamMember, Add-NetLbfoTeamNic, Get-NetLbfoTeam, Get-NetLbfoTeamMember...}
Manifest   1.0        NetLldpAgent                        {Enable-NetLldpAgent, Disable-NetLldpAgent, Get-NetLldpAgent}
Manifest   1.0.0.0    NetNat                              {Get-NetNat, Get-NetNatExternalAddress, Get-NetNatStaticMapping, Get-NetNatSession...}
Manifest   2.0.0.0    NetQos                              {Get-NetQosPolicy, Set-NetQosPolicy, Remove-NetQosPolicy, New-NetQosPolicy}
Manifest   2.0.0.0    NetSecurity                         {Get-DAPolicyChange, New-NetIPsecAuthProposal, New-NetIPsecMainModeCryptoProposal, New-NetIPsecQuickModeCryptoProposa...
Manifest   1.0.0.0    NetSwitchTeam                       {New-NetSwitchTeam, Remove-NetSwitchTeam, Get-NetSwitchTeam, Rename-NetSwitchTeam...}
Manifest   1.0.0.0    NetTCPIP                            {Get-NetIPAddress, Get-NetIPInterface, Get-NetIPv4Protocol, Get-NetIPv6Protocol...}
Manifest   1.0.0.0    NetworkConnectivityStatus           {Get-DAConnectionStatus, Get-NCSIPolicyConfiguration, Reset-NCSIPolicyConfiguration, Set-NCSIPolicyConfiguration}
Manifest   1.0.0.0    NetworkController                   {Add-NetworkControllerNode, Disable-NetworkControllerNode, Enable-NetworkControllerNode, Get-NetworkControllerCluster...
Manifest   1.0.0.0    NetworkControllerDiagnostics        {Get-NetworkControllerDeploymentInfo, Get-NetworkControllerManagedDevices, Debug-NetworkController, Debug-NetworkCont...
Manifest   2.0.0.0    NetworkLoadBalancingClusters        {Add-NlbClusterNode, Add-NlbClusterNodeDip, Add-NlbClusterPortRule, Add-NlbClusterVip...}
Manifest   1.0.0.0    NetworkSwitchManager                {Disable-NetworkSwitchEthernetPort, Enable-NetworkSwitchEthernetPort, Get-NetworkSwitchEthernetPort, Remove-NetworkSw...
Manifest   1.0.0.0    NetworkTransition                   {Add-NetIPHttpsCertBinding, Disable-NetDnsTransitionConfiguration, Disable-NetIPHttpsProfile, Disable-NetNatTransitio...
Manifest   1.0        NFS                                 {Get-NfsMappedIdentity, Get-NfsNetgroup, Install-NfsMappingStore, New-NfsMappedIdentity...}
Manifest   1.0.0.0    PcsvDevice                          {Get-PcsvDevice, Start-PcsvDevice, Stop-PcsvDevice, Restart-PcsvDevice...}
Binary     1.0.0.0    PersistentMemory                    {Get-PmemDisk, Get-PmemPhysicalDevice, Get-PmemUnusedRegion, New-PmemDisk...}
Manifest   1.0.0.0    PKI                                 {Add-CertificateEnrollmentPolicyServer, Export-Certificate, Export-PfxCertificate, Get-CertificateAutoEnrollmentPolic...
Manifest   1.0.0.0    PnpDevice                           {Get-PnpDevice, Get-PnpDeviceProperty, Enable-PnpDevice, Disable-PnpDevice}
Manifest   1.1        PrintManagement                     {Add-Printer, Add-PrinterDriver, Add-PrinterPort, Get-PrintConfiguration...}
Binary     1.0.12     ProcessMitigations                  {Get-ProcessMitigation, Set-ProcessMitigation, ConvertTo-ProcessMitigationPolicy}
Script     3.0        Provisioning                        {Install-ProvisioningPackage, Export-ProvisioningPackage, Install-TrustedProvisioningCertificate, Export-Trace...}
Manifest   1.1        PSDesiredStateConfiguration         {Set-DscLocalConfigurationManager, Start-DscConfiguration, Test-DscConfiguration, Publish-DscConfiguration...}
Script     1.0.0.0    PSDiagnostics                       {Disable-PSTrace, Disable-PSWSManCombinedTrace, Disable-WSManTrace, Enable-PSTrace...}
Binary     1.1.0.0    PSScheduledJob                      {New-JobTrigger, Add-JobTrigger, Remove-JobTrigger, Get-JobTrigger...}
Manifest   2.0.0.0    PSWorkflow                          {New-PSWorkflowExecutionOption, New-PSWorkflowSession, nwsn}
Manifest   1.0.0.0    PSWorkflowUtility                   Invoke-AsWorkflow
Manifest   3.0.0.0    RemoteAccess                        {Add-DAAppServer, Add-DAClient, Add-DAClientDnsConfiguration, Add-DAEntryPoint...}
Manifest   2.0.0.0    RemoteDesktop                       {Get-RDCertificate, Set-RDCertificate, New-RDCertificate, New-RDVirtualDesktopDeployment...}
Manifest   1.0.0.0    ScheduledTasks                      {Get-ScheduledTask, Set-ScheduledTask, Register-ScheduledTask, Unregister-ScheduledTask...}
Manifest   2.0.0.0    SecureBoot                          {Confirm-SecureBootUEFI, Set-SecureBootUEFI, Get-SecureBootUEFI, Format-SecureBootUEFI...}
Script     2.0.0.0    ServerManager                       {Get-WindowsFeature, Install-WindowsFeature, Uninstall-WindowsFeature, Enable-ServerManagerStandardUserRemoting...}
Cim        1.0.0.0    ServerManagerTasks                  {Get-SMCounterSample, Get-SMPerformanceCollector, Start-SMPerformanceCollector, Stop-SMPerformanceCollector...}
Manifest   1.0.0.0    ShieldedVMDataFile                  {Import-ShieldingDataFile, New-ShieldingDataFile, New-VolumeIDQualifier, Save-VolumeSignatureCatalog...}
Manifest   1.0.0.0    ShieldedVMTemplate                  {Protect-TemplateDisk, Initialize-VMShieldingHelperVHD}
Manifest   2.0.0.0    SmbShare                            {Get-SmbShare, Remove-SmbShare, Set-SmbShare, Block-SmbShareAccess...}
Manifest   2.0.0.0    SmbWitness                          {Get-SmbWitnessClient, Move-SmbWitnessClient, gsmbw, msmbw...}
Manifest   1.0.0.1    StartLayout                         {Export-StartLayout, Import-StartLayout, Export-StartLayoutEdgeAssets, Get-StartApps}
Manifest   2.0.0.0    Storage                             {Add-InitiatorIdToMaskingSet, Add-PartitionAccessPath, Add-PhysicalDisk, Add-StorageFaultDomain...}
Manifest   1.0.0.0    StorageBusCache                     {Clear-StorageBusDisk, Disable-StorageBusCache, Disable-StorageBusDisk, Enable-StorageBusCache...}
Manifest   1.0.0.0    StorageQoS                          {Get-StorageQoSPolicy, Get-StorageQoSPolicyStore, Set-StorageQoSPolicyStore, Remove-StorageQoSPolicy...}
Manifest   1.0        StorageReplica                      {Test-SRTopology, New-SRGroup, Remove-SRGroup, Set-SRGroup...}
Manifest   2.0.0.0    TLS                                 {New-TlsSessionTicketKey, Enable-TlsSessionTicketKey, Disable-TlsSessionTicketKey, Export-TlsSessionTicketKey...}
Manifest   1.0.0.0    TroubleshootingPack                 {Get-TroubleshootingPack, Invoke-TroubleshootingPack}
Manifest   2.0.0.0    TrustedPlatformModule               {Get-Tpm, Initialize-Tpm, Clear-Tpm, Unblock-Tpm...}
Binary     2.1.639.0  UEV                                 {Clear-UevConfiguration, Clear-UevAppxPackage, Restore-UevBackup, Set-UevTemplateProfile...}
Manifest   2.0.0.0    UpdateServices                      {Add-WsusComputer, Approve-WsusUpdate, Deny-WsusUpdate, Get-WsusClassification...}
Manifest   2.0.0.0    VpnClient                           {Add-VpnConnection, Set-VpnConnection, Remove-VpnConnection, Get-VpnConnection...}
Manifest   1.0.0.0    Wdac                                {Get-OdbcDriver, Set-OdbcDriver, Get-OdbcDsn, Add-OdbcDsn...}
Manifest   2.0.0.0    Whea                                {Get-WheaMemoryPolicy, Set-WheaMemoryPolicy}
Manifest   1.0.0.0    WindowsDeveloperLicense             {Get-WindowsDeveloperLicense, Unregister-WindowsDeveloperLicense, Show-WindowsDeveloperLicenseRegistration}
Script     1.0        WindowsErrorReporting               {Enable-WindowsErrorReporting, Disable-WindowsErrorReporting, Get-WindowsErrorReporting}
Manifest   1.0.0.0    WindowsSearch                       {Get-WindowsSearchSetting, Set-WindowsSearchSetting}
Manifest   1.0.0.0    WindowsUpdate                       Get-WindowsUpdateLog

    Directory: C:\Users\GAS02047\.vscode-insiders\extensions\ms-vscode.powershell-2021.9.0\modules

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     1.1.3      Plaster                             {Invoke-Plaster, New-PlasterManifest, Get-PlasterTemplate, Test-PlasterManifest}
Binary     2.5.1      PowerShellEditorServices            Start-EditorServices
Binary     0.2.0      PowerShellEditorServices.VSCode     {New-VSCodeHtmlContentView, Show-VSCodeHtmlContentView, Close-VSCodeHtmlContentView, Set-VSCodeHtmlContentView...}
Script     2.1.0      PSReadLine                          {Get-PSReadLineKeyHandler, Set-PSReadLineKeyHandler, Remove-PSReadLineKeyHandler, Get-PSReadLineOption...}
Script     1.20.0     PSScriptAnalyzer                    {Get-ScriptAnalyzerRule, Invoke-ScriptAnalyzer, Invoke-Formatter}

Debug output

Output when it works from the point where I output the contents of the $Confg variable:

{
    "TargetServer":  "dwqa.gassouth.com",
    "TargetInstance":  "MSSQLSERVER",
    "RestoredDBName":  "GSDWCON",
    "TargetVirtualMachineId":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA",
    "OverwriteWLIfpresent":  "Yes",
    "NoRecoveryMode":  "Disabled",
    "targetPhysicalPath":  [
                               {
                                   "MappingType":  "Data",
                                   "SourceLogicalName":  "GSDWCON",
                                   "SourcePath":  "R:\\Data\\GSDWCON.mdf",
                                   "TargetPath":  "E:\\SQLData\\GSDWCON\\GSDWCON.mdf"
                               },
                               {
                                   "MappingType":  "Log",
                                   "SourceLogicalName":  "GSDWCON_log",
                                   "SourcePath":  "Z:\\Log\\GSDWCON_log.ldf",
                                   "TargetPath":  "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf"
                               }
                           ],
    "ContainerId":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa",
    "FullRP":  null,
    "SourceResourceId":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01",
    "RestoreRequestType":  "Alternate WL Restore to diff item",
    "RecoveryPoint":  {
                          "RecoveryPointTier":  0,
                          "RecoveryPointMoveReadinessInfo":  null,
                          "RehydrationExpiryTime":  null,
                          "DataDirectoryPaths":  null,
                          "RecoveryPointType":  null,
                          "RecoveryPointTime":  "\/Date(-62135578800000)\/",
                          "RecoveryPointId":  "DefaultRangeRecoveryPoint",
                          "ItemName":  null,
                          "Id":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint",
                          "WorkloadType":  0,
                          "ContainerName":  null,
                          "ContainerType":  0,
                          "BackupManagementType":  0
                      },
    "PointInTime":  "\/Date(1632756730000)\/",
    "RecoveryMode":  null,
    "FilePath":  null
}
DEBUG: 1:59:04 PM - RestoreAzureRmRecoveryServicesBackupItem begin processing with ParameterSet 'AzureWorkloadParameterSet'.
DEBUG: 1:59:04 PM - using account id 'GAS02047@GasSouth.com'...
DEBUG: Inside Restore. Going to create ResourceClient.
DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec'
DEBUG: 1:59:04 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Found 1 cache accounts and 0 broker accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Returning 1 accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(63d69379-1e96-4b55-abe5-f46777b338c0)
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === AcquireTokenSilent Parameters ===
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] LoginHint provided: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Account provided: True
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] ForceRefresh: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ]
=== Request Data ===
Authority Provided? - True
Scopes - https://management.core.windows.net//.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - 63d69379-1e96-4b55-abe5-f46777b338c0

DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition (SilentRequest) started:

        Authority Host: login.microsoftonline.com
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 17:59:04) - Expiration Time (09/27/2021 18:33:33 +00:00) - Extended Expiration Time (09/27/2021 18:33:33 +00:00)]
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Returning access token found in cache. RefreshOn exists ? False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Fetched access token from host login.microsoftonline.com.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition finished successfully. An access token was returned with Expiration
Time: 09/27/2021 18:33:33 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:  ExpiresOn: 2021-09-27T18:33:33.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com'
DEBUG: Client Created successfully
DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec'
DEBUG: 1:59:04 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Found 1 cache accounts and 0 broker accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Returning 1 accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(8fbbef67-e99e-43a2-83b8-87f3ad3e0290)
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === AcquireTokenSilent Parameters ===
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] LoginHint provided: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Account provided: True
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] ForceRefresh: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ]
=== Request Data ===
Authority Provided? - True
Scopes - https://management.core.windows.net//.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - 8fbbef67-e99e-43a2-83b8-87f3ad3e0290

DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition (SilentRequest) started:

        Authority Host: login.microsoftonline.com
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 17:59:04) - Expiration Time (09/27/2021 18:33:33 +00:00) - Extended Expiration Time (09/27/2021 18:33:33 +00:00)]
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Returning access token found in cache. RefreshOn exists ? False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Fetched access token from host login.microsoftonline.com.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition finished successfully. An access token was returned with Expiration
Time: 09/27/2021 18:33:33 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:  ExpiresOn: 2021-09-27T18:33:33.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com'
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint/restore?api-version=2021-06-01

Headers:
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d
accept-language               : en-US

Body:
{
  "properties": {
    "objectType": "AzureWorkloadSQLPointInTimeRestoreRequest",
    "pointInTime": "2021-09-27T15:32:10Z",
    "shouldUseAlternateTargetLocation": true,
    "isNonRecoverable": false,
    "alternateDirectoryPaths": [
      {
        "mappingType": "Data",
        "sourceLogicalName": "GSDWCON",
        "sourcePath": "R:\\Data\\GSDWCON.mdf",
        "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON.mdf"
      },
      {
        "mappingType": "Log",
        "sourceLogicalName": "GSDWCON_log",
        "sourcePath": "Z:\\Log\\GSDWCON_log.ldf",
        "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf"
      }
    ],
    "recoveryType": "AlternateLocation",
    "sourceResourceId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01",
    "targetInfo": {
      "overwriteOption": "Overwrite",
      "containerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa",
      "databaseName": "MSSQLSERVER/GSDWCON"
    },
    "targetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA"
  }
}

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
Accepted

Headers:
Pragma                        : no-cache
Retry-After                   : 60
Azure-AsyncOperation          : https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01
X-Content-Type-Options        : nosniff
x-ms-request-id               : 6e03b616-cb10-43a0-a781-e5301876ea15
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-writes: 1199
x-ms-correlation-request-id   : 6e03b616-cb10-43a0-a781-e5301876ea15
x-ms-routing-request-id       : CENTRALUS:20210927T175906Z:6e03b616-cb10-43a0-a781-e5301876ea15
Cache-Control                 : no-cache
Date                          : Mon, 27 Sep 2021 17:59:05 GMT
Location                      : https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationResults/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01
X-Powered-By                  : ASP.NET

Body:

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01

Headers:
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Pragma                        : no-cache
X-Content-Type-Options        : nosniff
x-ms-request-id               : 665a97ef-af5d-45ac-bf77-6875ecd27aec
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-resource-requests: 149
x-ms-correlation-request-id   : 665a97ef-af5d-45ac-bf77-6875ecd27aec
x-ms-routing-request-id       : CENTRALUS:20210927T180006Z:665a97ef-af5d-45ac-bf77-6875ecd27aec
Cache-Control                 : no-cache
Date                          : Mon, 27 Sep 2021 18:00:06 GMT
Server                        : Microsoft-IIS/10.0
X-Powered-By                  : ASP.NET

Body:
{
  "id": "dc370263-0e68-4ed9-b266-5a84c6731e06",
  "name": "dc370263-0e68-4ed9-b266-5a84c6731e06",
  "status": "Succeeded",
  "startTime": "2021-09-27T17:59:05.7299304Z",
  "endTime": "2021-09-27T17:59:05.7299304Z",
  "properties": {
    "objectType": "OperationStatusJobExtendedInfo",
    "jobId": "dc370263-0e68-4ed9-b266-5a84c6731e06"
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationResults/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01

Headers:
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
NoContent

Headers:
Pragma                        : no-cache
Retry-After                   : 60
Azure-AsyncOperation          : https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01
X-Content-Type-Options        : nosniff
x-ms-request-id               : 95621390-59e5-4c14-99d9-b1107af1bf49
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-resource-requests: 149
x-ms-correlation-request-id   : 95621390-59e5-4c14-99d9-b1107af1bf49
x-ms-routing-request-id       : CENTRALUS:20210927T180007Z:95621390-59e5-4c14-99d9-b1107af1bf49
Cache-Control                 : no-cache
Date                          : Mon, 27 Sep 2021 18:00:07 GMT
X-Powered-By                  : ASP.NET

Body:

DEBUG: Tracking operation status URL for completion:https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupOperations/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01

Headers:
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d
accept-language               : en-US

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Pragma                        : no-cache
X-Content-Type-Options        : nosniff
x-ms-request-id               : d0091d1c-cc84-4a54-90fc-57e36fe56279
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-resource-requests: 149
x-ms-correlation-request-id   : d0091d1c-cc84-4a54-90fc-57e36fe56279
x-ms-routing-request-id       : CENTRALUS:20210927T180008Z:d0091d1c-cc84-4a54-90fc-57e36fe56279
Cache-Control                 : no-cache
Date                          : Mon, 27 Sep 2021 18:00:07 GMT
Server                        : Microsoft-IIS/10.0
X-Powered-By                  : ASP.NET

Body:
{
  "id": "dc370263-0e68-4ed9-b266-5a84c6731e06",
  "name": "dc370263-0e68-4ed9-b266-5a84c6731e06",
  "status": "Succeeded",
  "startTime": "2021-09-27T17:59:05.7299304Z",
  "endTime": "2021-09-27T17:59:05.7299304Z",
  "properties": {
    "objectType": "OperationStatusJobExtendedInfo",
    "jobId": "dc370263-0e68-4ed9-b266-5a84c6731e06"
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupOperations/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01

Headers:
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d
accept-language               : en-US

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Pragma                        : no-cache
X-Content-Type-Options        : nosniff
x-ms-request-id               : 37362667-662b-40f5-aa0d-c9e1e49b61e2
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-resource-requests: 148
x-ms-correlation-request-id   : 37362667-662b-40f5-aa0d-c9e1e49b61e2
x-ms-routing-request-id       : CENTRALUS:20210927T180008Z:37362667-662b-40f5-aa0d-c9e1e49b61e2
Cache-Control                 : no-cache
Date                          : Mon, 27 Sep 2021 18:00:08 GMT
Server                        : Microsoft-IIS/10.0
X-Powered-By                  : ASP.NET

Body:
{
  "id": "dc370263-0e68-4ed9-b266-5a84c6731e06",
  "name": "dc370263-0e68-4ed9-b266-5a84c6731e06",
  "status": "Succeeded",
  "startTime": "2021-09-27T17:59:05.7299304Z",
  "endTime": "2021-09-27T17:59:05.7299304Z",
  "properties": {
    "objectType": "OperationStatusJobExtendedInfo",
    "jobId": "dc370263-0e68-4ed9-b266-5a84c6731e06"
  }
}

DEBUG: Final operation status:Succeeded
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupJobs/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01

Headers:
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d
accept-language               : en-US

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Pragma                        : no-cache
Cache-Control                 : no-cache
Server                        : Microsoft-IIS/10.0,Microsoft-IIS/10.0
X-Content-Type-Options        : nosniff
x-ms-request-id               : 35c9bfcb-d282-4fb1-a833-bf5f99106748
x-ms-client-request-id        : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d
X-Powered-By                  : ASP.NET
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-resource-requests: 149
x-ms-correlation-request-id   : 35c9bfcb-d282-4fb1-a833-bf5f99106748
x-ms-routing-request-id       : CENTRALUS:20210927T180008Z:35c9bfcb-d282-4fb1-a833-bf5f99106748
Date                          : Mon, 27 Sep 2021 18:00:08 GMT

Body:
{
  "id": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupJobs/dc370263-0e68-4ed9-b266-5a84c6731e06",
  "name": "dc370263-0e68-4ed9-b266-5a84c6731e06",
  "type": "Microsoft.RecoveryServices/vaults/backupJobs",
  "properties": {
    "jobType": "AzureWorkloadJob",
    "actionsInfo": [
      1
    ],
    "workloadType": "SQLDataBase",
    "duration": "PT1M3.0147601S",
    "extendedInfo": {
      "tasksList": [
        {
          "taskId": "Transfer data from vault",
          "status": "InProgress"
        }
      ],
      "propertyBag": {
        "Job Type": "Recovery to an alternate database"
      }
    },
    "isUserTriggered": true,
    "entityFriendlyName": "GSDWCON [dwprod01.gassouth.com]",
    "backupManagementType": "AzureWorkload",
    "operation": "Restore",
    "status": "InProgress",
    "startTime": "2021-09-27T17:59:05.7299304Z",
    "activityId": "77b77826-02eb-438c-a92f-84a78c570a3d"
  }
}

WorkloadName     Operation            Status               StartTime                 EndTime                   JobID
------------     ---------            ------               ---------                 -------                   -----
GSDWCON [dwpr... Restore              InProgress           9/27/2021 5:59:05 PM                                dc370263-0e68-4ed9-b266-5a84c6731e06
DEBUG: AzureQoSEvent: Module: Az.RecoveryServices:4.6.0; CommandName: Restore-AzRecoveryServicesBackupItem; PSVersion: 5.1.19041.1237; IsSuccess: True; Duration: 00:01:04.2375683
DEBUG: Finish sending metric.
DEBUG: 2:00:09 PM - RestoreAzureRmRecoveryServicesBackupItem end processing.

Output when it fails:

{
    "TargetServer":  "dwqa.gassouth.com",
    "TargetInstance":  "MSSQLSERVER",
    "RestoredDBName":  "GSDWCON",
    "TargetVirtualMachineId":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA",
    "OverwriteWLIfpresent":  "Yes",
    "NoRecoveryMode":  "Disabled",
    "targetPhysicalPath":  [
                               {
                                   "MappingType":  "Data",
                                   "SourceLogicalName":  "GSDWCON",
                                   "SourcePath":  "R:\\Data\\GSDWCON.mdf",
                                   "TargetPath":  "E:\\SQLData\\GSDWCON\\GSDWCON.mdf"
                               },
                               {
                                   "MappingType":  "Log",
                                   "SourceLogicalName":  "GSDWCON_log",
                                   "SourcePath":  "Z:\\Log\\GSDWCON_log.ldf",
                                   "TargetPath":  "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf"
                               }
                           ],
    "ContainerId":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa",
    "FullRP":  null,
    "SourceResourceId":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01",
    "RestoreRequestType":  "Alternate WL Restore to diff item",
    "RecoveryPoint":  {
                          "RecoveryPointTier":  0,
                          "RecoveryPointMoveReadinessInfo":  null,
                          "RehydrationExpiryTime":  null,
                          "DataDirectoryPaths":  null,
                          "RecoveryPointType":  null,
                          "RecoveryPointTime":  "\/Date(-62135578800000)\/",
                          "RecoveryPointId":  "DefaultRangeRecoveryPoint",
                          "ItemName":  null,
                          "Id":  "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint",
                          "WorkloadType":  0,
                          "ContainerName":  null,
                          "ContainerType":  0,
                          "BackupManagementType":  0
                      },
    "PointInTime":  "\/Date(1632771248000)\/",
    "RecoveryMode":  null,
    "FilePath":  null
}
DEBUG: 3:42:25 PM - RestoreAzureRmRecoveryServicesBackupItem begin processing with ParameterSet 'AzureWorkloadParameterSet'.
DEBUG: 3:42:25 PM - using account id 'GAS02047@GasSouth.com'...
DEBUG: Inside Restore. Going to create ResourceClient.
DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec'
DEBUG: 3:42:25 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Found 1 cache accounts and 0 broker accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Returning 1 accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(e0eb4966-afbb-465a-9b60-cf231b31fddb)
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === AcquireTokenSilent Parameters ===
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] LoginHint provided: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Account provided: True
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] ForceRefresh: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ]
=== Request Data ===
Authority Provided? - True
Scopes - https://management.core.windows.net//.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - e0eb4966-afbb-465a-9b60-cf231b31fddb

DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition (SilentRequest) started:

        Authority Host: login.microsoftonline.com
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 19:42:25) - Expiration Time (09/27/2021 20:12:03 +00:00) - Extended Expiration Time (09/27/2021 20:12:03 +00:00)]
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Returning access token found in cache. RefreshOn exists ? False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Fetched access token from host login.microsoftonline.com.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition finished successfully. An access token was returned with Expiration
Time: 09/27/2021 20:12:03 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:  ExpiresOn: 2021-09-27T20:12:03.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com'
DEBUG: Client Created successfully
DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec'
DEBUG: 3:42:25 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Found 1 cache accounts and 0 broker accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Returning 1 accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(3ad726ae-cf5b-4c45-ae8b-8363ad62ecbd)
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === AcquireTokenSilent Parameters ===
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] LoginHint provided: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Account provided: True
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] ForceRefresh: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ]
=== Request Data ===
Authority Provided? - True
Scopes - https://management.core.windows.net//.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - 3ad726ae-cf5b-4c45-ae8b-8363ad62ecbd

DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition (SilentRequest) started:

        Authority Host: login.microsoftonline.com
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 19:42:25) - Expiration Time (09/27/2021 20:12:03 +00:00) - Extended Expiration Time (09/27/2021 20:12:03 +00:00)]
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Returning access token found in cache. RefreshOn exists ? False
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Fetched access token from host login.microsoftonline.com.
DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition finished successfully. An access token was returned with Expiration
Time: 09/27/2021 20:12:03 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:  ExpiresOn: 2021-09-27T20:12:03.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com'
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint/restore?api-version=2021-06-01

Headers:
x-ms-client-request-id        : 36b8096b-73f3-4c91-a567-df2948237d2e
accept-language               : en-US

Body:
{
  "properties": {
    "objectType": "AzureWorkloadSQLPointInTimeRestoreRequest",
    "pointInTime": "2021-09-27T19:34:08Z",
    "shouldUseAlternateTargetLocation": true,
    "isNonRecoverable": false,
    "alternateDirectoryPaths": [
      {
        "mappingType": "Data",
        "sourceLogicalName": "GSDWCON",
        "sourcePath": "R:\\Data\\GSDWCON.mdf",
        "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON.mdf"
      },
      {
        "mappingType": "Log",
        "sourceLogicalName": "GSDWCON_log",
        "sourcePath": "Z:\\Log\\GSDWCON_log.ldf",
        "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf"
      }
    ],
    "recoveryType": "AlternateLocation",
    "sourceResourceId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01",
    "targetInfo": {
      "overwriteOption": "Overwrite",
      "containerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa",
      "databaseName": "MSSQLSERVER/GSDWCON"
    },
    "targetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA"
  }
}

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
BadRequest

Headers:
Pragma                        : no-cache
X-Content-Type-Options        : nosniff
x-ms-request-id               : 0eb7b15f-642b-4f94-bcd6-a63048b32a5c
x-ms-client-request-id        : 36b8096b-73f3-4c91-a567-df2948237d2e,36b8096b-73f3-4c91-a567-df2948237d2e
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-writes: 1199
x-ms-correlation-request-id   : 0eb7b15f-642b-4f94-bcd6-a63048b32a5c
x-ms-routing-request-id       : CENTRALUS:20210927T194226Z:0eb7b15f-642b-4f94-bcd6-a63048b32a5c
Cache-Control                 : no-cache
Date                          : Mon, 27 Sep 2021 19:42:26 GMT
Server                        : Microsoft-IIS/10.0
X-Powered-By                  : ASP.NET

Body:
{
  "error": {
    "code": "BMSUserErrorInvalidInput",
    "message": "Input provided for the call is invalid. Please check the required inputs",
    "target": null,
    "details": null,
    "innerError": null
  }
}

DEBUG: Caught exception, type: System.AggregateException
DEBUG: Handling aggregate exception
DEBUG: Received CloudException, ErrorCode: BMSUserErrorInvalidInput, Message: Input provided for the call is invalid. Please check the required inputs
Restore-AzRecoveryServicesBackupItem : Input provided for the call is invalid. Please check the required inputs
At C:\Users\GAS02047\OneDrive - Gas South\Documents\testcase-Restore-AzRecoveryServicesBackupItem.ps1:14 char:1
+ Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Restore-AzRecoveryServicesBackupItem], CloudException
    + FullyQualifiedErrorId : BMSUserErrorInvalidInput,Microsoft.Azure.Commands.RecoveryServices.Backup.Cmdlets.RestoreAzureRmRecoveryServicesBackupItem

DEBUG: AzureQoSEvent: Module: Az.RecoveryServices:4.6.0; CommandName: Restore-AzRecoveryServicesBackupItem; PSVersion: 5.1.19041.1237; IsSuccess: False; Duration: 00:00:01.1917939; Exception: Input provided for the call is invalid. Please check the required inputs;
DEBUG: Finish sending metric.
DEBUG: 3:42:27 PM - RestoreAzureRmRecoveryServicesBackupItem end processing.

Error output

   HistoryId: 18

RequestId      : 0eb7b15f-642b-4f94-bcd6-a63048b32a5c
Message        : Input provided for the call is invalid. Please check the required inputs
ServerMessage  : BMSUserErrorInvalidInput: Input provided for the call is invalid. Please check the required inputs
                 (System.Collections.Generic.List`1[Microsoft.Rest.Azure.CloudError])
ServerResponse : {BadRequest}
RequestMessage : {POST https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/
                 vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/reco
                 veryPoints/DefaultRangeRecoveryPoint/restore?api-version=2021-06-01}
InvocationInfo : {Restore-AzRecoveryServicesBackupItem}
Line           : Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config

Position       : At C:\Users\GAS02047\OneDrive - Gas South\Documents\testcase-Restore-AzRecoveryServicesBackupItem.ps1:14 char:1
                 + Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config
                 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
StackTrace     :    at Microsoft.Azure.Management.RecoveryServices.Backup.RestoresOperations.<BeginTriggerWithHttpMessagesAsync>d__6.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at Microsoft.Azure.Management.RecoveryServices.Backup.RestoresOperations.<TriggerWithHttpMessagesAsync>d__5.MoveNext()
HistoryId      : 18
mwsutherland commented 3 years ago

Since I was still in a state where things were failing, I made a change to the code to subtract an hour from the last LastRecoveryPoint (the log backups are every 4 hours) and it worked. Trying again with the unmodified timestamp failed. Trying a 3rd time with the timestamp reduced by 5 minutes also worked.

So, I think I've found something here. Maybe the consistent failure of the 1st of a pair of restores was just a coincidence My sample size was only about 3 attempts on different nights.

mwsutherland commented 3 years ago

I just looked at the LastRecoveryPoint of my failed run. It was an even second. If point in time is measured in fractions of a second and that is being rounded to the nearest second for the LastRecoveryPoint field, half of the time that value is going to be invalid.

mwsutherland commented 3 years ago

I let my two-database restore script run tonight with a 5 minute reduction applied to LastRecoveryPoint. The first succeeded and the 2nd failed, which is the opposite of the previous attempts. So, I guess that kills any thought that the order matters and puts a significant hurt on my theory that the problem is rounding of LastRecoveryPoint.

ghost commented 3 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @pvrk, @adityabalaji-msft.

Issue Details
## Description Sometimes Restore-AzRecoveryServicesBackupItem fails for no obvious reason. I've had the same database sometimes fail and sometimes succeed. My intended use case has me trying to restore two databases. The first fails and the second succeeds. If I swap the order, the new first fails and the new second succeeds. In testing with a single database, I find that it will consistently either fail or succeed for quite a while. I started working on this ticket on a Friday. Everything was failing as I put together my demonstration script. The work day was over and when I ran it on Monday to gather output, it worked. The good news is that, after waiting for a few hours, it failed again, so now I have output for both cases. Comparing the output, the only thing I notice different before the error, other than time stamps and correlation IDs, is that another log backup had happened between the two attempts, so they aren't actually targeting the same point-in-time. I don't know why that should matter unless the LatestRecoveryPoint field of the backup I'm trying to restore isn't always valid point-in-time to use. That doesn't seem to explain my scenario where I swap the order of restores, but I'll investigate this angle further and add any new findings in the comments. ## Steps to reproduce ```powershell Connect-AzAccount Set-AzContext "Microsoft Azure Enterprise" $Vault = Get-AzRecoveryServicesVault -ResourceGroupName "New-LRS-Recovery-Vault" -Name "New-LRS-Recovery-Vault" Set-AzRecoveryServicesVaultContext -Vault $Vault $BackupItem = Get-AzRecoveryServicesBackupItem -BackupManagementType "AzureWorkload" -WorkloadType "MSSQL" -Name "SQLDataBase;MSSQLSERVER;GSDWCON" $TargetInstance = Get-AzRecoveryServicesBackupProtectableItem -WorkloadType "MSSQL" -ItemType "SQLInstance" -ServerName "dwqa.gassouth.com" -Name "sqlinstance;mssqlserver" $TargetContainer = Get-AzRecoveryServicesBackupContainer -ContainerType "AzureVMAppContainer" -FriendlyName "DWQA" $Config = Get-AzRecoveryServicesBackupWorkloadRecoveryConfig -Item $BackupItem -PointInTime $BackupItem.LatestRecoveryPoint -TargetItem $TargetInstance -TargetContainer $TargetContainer -AlternateWorkloadRestore $Config.RestoredDBName = "GSDWCON" $Config.OverwriteWLIfpresent = "Yes" $Config.targetPhysicalPath[0].TargetPath = "E:\SQLData\GSDWCON\GSDWCON.mdf" $Config.targetPhysicalPath[1].TargetPath = "E:\SQLData\GSDWCON\GSDWCON_log.ldf" # Display contents of parameter $Config | ConvertTo-Json Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config ``` ## Environment data ``` Name Value ---- ----- PSVersion 5.1.19041.1237 PSEdition Desktop PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...} BuildVersion 10.0.19041.1237 CLRVersion 4.0.30319.42000 WSManStackVersion 3.0 PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 ``` ## Module versions ```powershell Directory: C:\Program Files\WindowsPowerShell\Modules ModuleType Version Name ExportedCommands ---------- ------- ---- ---------------- Script 2.5.3 Az.Accounts {Disable-AzDataCollection, Disable-AzContextAutosave, Enable-AzDataCollection, Enable-AzContextAutosave...} Script 1.1.1 Az.Advisor {Get-AzAdvisorRecommendation, Enable-AzAdvisorRecommendation, Disable-AzAdvisorRecommendation, Get-AzAdvisorConfigura... Script 2.4.0 Az.Aks {Get-AzAksCluster, New-AzAksCluster, Remove-AzAksCluster, Import-AzAksCredential...} Script 1.1.4 Az.AnalysisServices {Resume-AzAnalysisServicesServer, Suspend-AzAnalysisServicesServer, Get-AzAnalysisServicesServer, Remove-AzAnalysisSe... Script 2.3.0 Az.ApiManagement {Add-AzApiManagementApiToGateway, Add-AzApiManagementApiToProduct, Add-AzApiManagementProductToGroup, Add-AzApiManage... Script 1.0.0 Az.AppConfiguration {Get-AzAppConfigurationStore, Get-AzAppConfigurationStoreKey, New-AzAppConfigurationStore, New-AzAppConfigurationStor... Script 1.2.0 Az.ApplicationInsights {Get-AzApplicationInsights, New-AzApplicationInsights, Remove-AzApplicationInsights, Update-AzApplicationInsights...} Script 1.7.1 Az.Automation {Get-AzAutomationHybridWorkerGroup, Remove-AzAutomationHybridWorkerGroup, Get-AzAutomationJobOutputRecord, Import-AzA... Script 3.1.0 Az.Batch {Remove-AzBatchAccount, Get-AzBatchAccount, Get-AzBatchAccountKey, New-AzBatchAccount...} Script 2.0.0 Az.Billing {Get-AzBillingInvoice, Get-AzBillingPeriod, Get-AzEnrollmentAccount, Get-AzConsumptionBudget...} Script 1.8.0 Az.Cdn {Get-AzCdnProfile, Get-AzCdnProfileSsoUrl, New-AzCdnProfile, Remove-AzCdnProfile...} Script 1.9.0 Az.CognitiveServices {Get-AzCognitiveServicesAccount, Get-AzCognitiveServicesAccountKey, Get-AzCognitiveServicesAccountSku, Get-AzCognitiv... Script 4.17.0 Az.Compute {Remove-AzAvailabilitySet, Get-AzAvailabilitySet, New-AzAvailabilitySet, Update-AzAvailabilitySet...} Script 2.1.0 Az.ContainerInstance {Add-AzContainerInstanceOutput, Get-AzContainerGroup, Get-AzContainerInstanceCachedImage, Get-AzContainerInstanceCapa... Script 2.2.3 Az.ContainerRegistry {New-AzContainerRegistry, Get-AzContainerRegistry, Update-AzContainerRegistry, Remove-AzContainerRegistry...} Script 1.3.1 Az.CosmosDB {Get-AzCosmosDBSqlContainer, Get-AzCosmosDBSqlContainerThroughput, Get-AzCosmosDBSqlDatabase, Get-AzCosmosDBSqlDataba... Script 1.1.0 Az.DataBoxEdge {Get-AzDataBoxEdgeJob, Get-AzDataBoxEdgeDevice, Invoke-AzDataBoxEdgeDevice, New-AzDataBoxEdgeDevice...} Script 1.1.0 Az.Databricks {Get-AzDatabricksVNetPeering, Get-AzDatabricksWorkspace, New-AzDatabricksVNetPeering, New-AzDatabricksWorkspace...} Script 1.14.0 Az.DataFactory {Set-AzDataFactoryV2, Update-AzDataFactoryV2, Get-AzDataFactoryV2, Remove-AzDataFactoryV2...} Script 1.0.2 Az.DataLakeAnalytics {Get-AzDataLakeAnalyticsDataSource, New-AzDataLakeAnalyticsCatalogCredential, Remove-AzDataLakeAnalyticsCatalogCreden... Script 1.3.0 Az.DataLakeStore {Get-AzDataLakeStoreTrustedIdProvider, Remove-AzDataLakeStoreTrustedIdProvider, Remove-AzDataLakeStoreFirewallRule, S... Script 1.0.0 Az.DataShare {New-AzDataShareAccount, Get-AzDataShareAccount, Remove-AzDataShareAccount, New-AzDataShare...} Script 1.1.0 Az.DeploymentManager {Get-AzDeploymentManagerArtifactSource, New-AzDeploymentManagerArtifactSource, Set-AzDeploymentManagerArtifactSource,... Script 3.0.0 Az.DesktopVirtualization {Disconnect-AzWvdUserSession, Expand-AzWvdMsixImage, Get-AzWvdApplication, Get-AzWvdApplicationGroup...} Script 1.0.2 Az.DevTestLabs {Get-AzDtlAllowedVMSizesPolicy, Get-AzDtlAutoShutdownPolicy, Get-AzDtlAutoStartPolicy, Get-AzDtlVMsPerLabPolicy...} Script 1.1.2 Az.Dns {Get-AzDnsRecordSet, New-AzDnsRecordConfig, Remove-AzDnsRecordSet, Set-AzDnsRecordSet...} Script 1.3.0 Az.EventGrid {New-AzEventGridTopic, Get-AzEventGridTopic, Set-AzEventGridTopic, New-AzEventGridTopicKey...} Script 1.8.0 Az.EventHub {New-AzEventHubNamespace, Get-AzEventHubNamespace, Set-AzEventHubNamespace, Remove-AzEventHubNamespace...} Script 1.8.0 Az.FrontDoor {New-AzFrontDoor, Get-AzFrontDoor, Set-AzFrontDoor, Remove-AzFrontDoor...} Script 3.1.0 Az.Functions {Get-AzFunctionApp, Get-AzFunctionAppAvailableLocation, Get-AzFunctionAppPlan, Get-AzFunctionAppSetting...} Script 4.3.0 Az.HDInsight {Get-AzHDInsightJob, New-AzHDInsightSqoopJobDefinition, Wait-AzHDInsightJob, New-AzHDInsightStreamingMapReduceJobDefi... Script 1.3.1 Az.HealthcareApis {New-AzHealthcareApisService, Remove-AzHealthcareApisService, Set-AzHealthcareApisService, Get-AzHealthcareApisService} Script 2.7.3 Az.IotHub {Add-AzIotHubKey, Get-AzIotHubEventHubConsumerGroup, Get-AzIotHubConnectionString, Get-AzIotHubJob...} Script 3.5.0 Az.KeyVault {Add-AzKeyVaultCertificate, Update-AzKeyVaultCertificate, Stop-AzKeyVaultCertificateOperation, Get-AzKeyVaultCertific... Script 2.0.0 Az.Kusto {Add-AzKustoClusterLanguageExtension, Add-AzKustoDatabasePrincipal, Get-AzKustoAttachedDatabaseConfiguration, Get-AzK... Script 1.5.0 Az.LogicApp {Get-AzIntegrationAccountAgreement, Get-AzIntegrationAccountAssembly, Get-AzIntegrationAccountBatchConfiguration, Get... Script 1.1.3 Az.MachineLearning {Move-AzMlCommitmentAssociation, Get-AzMlCommitmentAssociation, Get-AzMlCommitmentPlanUsageHistory, Remove-AzMlCommit... Script 1.1.1 Az.Maintenance {Get-AzApplyUpdate, Get-AzConfigurationAssignment, Get-AzMaintenanceConfiguration, Get-AzMaintenanceUpdate...} Script 2.0.0 Az.ManagedServices {Get-AzManagedServicesAssignment, New-AzManagedServicesAssignment, Remove-AzManagedServicesAssignment, Get-AzManagedS... Script 1.0.2 Az.MarketplaceOrdering {Get-AzMarketplaceTerms, Set-AzMarketplaceTerms} Script 1.1.1 Az.Media {Sync-AzMediaServiceStorageKey, Set-AzMediaServiceKey, Get-AzMediaServiceKey, Get-AzMediaServiceNameAvailability...} Script 1.1.1 Az.Migrate {Get-AzMigrateDiscoveredServer, Get-AzMigrateJob, Get-AzMigrateProject, Get-AzMigrateReplicationEligibilityResult...} Script 2.7.0 Az.Monitor {Get-AzMetricDefinition, Get-AzMetric, Remove-AzLogProfile, Get-AzLogProfile...} Script 0.7.0 Az.NetAppFiles {Get-AzNetAppFilesAccount, New-AzNetAppFilesAccount, Remove-AzNetAppFilesAccount, Set-AzNetAppFilesAccount...} Script 4.11.0 Az.Network {Add-AzApplicationGatewayAuthenticationCertificate, Get-AzApplicationGatewayAuthenticationCertificate, New-AzApplicat... Script 1.1.1 Az.NotificationHubs {Get-AzNotificationHub, Get-AzNotificationHubAuthorizationRule, Get-AzNotificationHubListKey, Get-AzNotificationHubPN... Script 2.3.0 Az.OperationalInsights {New-AzOperationalInsightsAzureActivityLogDataSource, New-AzOperationalInsightsCustomLogDataSource, Disable-AzOperati... Script 1.4.1 Az.PolicyInsights {Get-AzPolicyEvent, Get-AzPolicyState, Get-AzPolicyStateSummary, Get-AzPolicyRemediation...} Script 1.1.2 Az.PowerBIEmbedded {Remove-AzPowerBIWorkspaceCollection, Get-AzPowerBIWorkspaceCollection, Get-AzPowerBIWorkspaceCollectionAccessKey, Ge... Script 1.0.3 Az.PrivateDns {Get-AzPrivateDnsZone, Remove-AzPrivateDnsZone, Set-AzPrivateDnsZone, New-AzPrivateDnsZone...} Script 4.6.0 Az.RecoveryServices {Get-AzRecoveryServicesBackupProperty, Get-AzRecoveryServicesVault, Get-AzRecoveryServicesVaultSettingsFile, New-AzRe... Script 1.5.0 Az.RedisCache {Remove-AzRedisCachePatchSchedule, New-AzRedisCacheScheduleEntry, Get-AzRedisCachePatchSchedule, New-AzRedisCachePatc... Script 1.0.0 Az.RedisEnterpriseCache {Export-AzRedisEnterpriseCache, Get-AzRedisEnterpriseCache, Get-AzRedisEnterpriseCacheDatabase, Get-AzRedisEnterprise... Script 1.0.3 Az.Relay {New-AzRelayNamespace, Get-AzRelayNamespace, Set-AzRelayNamespace, Remove-AzRelayNamespace...} Script 0.9.0 Az.Reservations {Get-AzReservationOrder, Get-AzReservation, Get-AzReservationHistory, Update-AzReservation...} Script 1.0.0 Az.ResourceMover {Add-AzResourceMoverMoveResource, Get-AzResourceMoverMoveCollection, Get-AzResourceMoverMoveResource, Get-AzResourceM... Script 4.3.1 Az.Resources {Get-AzProviderOperation, Remove-AzRoleAssignment, Get-AzRoleAssignment, New-AzRoleAssignment...} Script 1.0.0 Az.Security {Get-AzSecurityAlert, Set-AzSecurityAlert, Get-AzSecurityAutoProvisioningSetting, Set-AzSecurityAutoProvisioningSetti... Script 1.1.0 Az.SecurityInsights {Get-AzSentinelAlertRuleAction, New-AzSentinelAlertRuleAction, Remove-AzSentinelAlertRuleAction, Update-AzSentinelAle... Script 1.5.0 Az.ServiceBus {New-AzServiceBusNamespace, Get-AzServiceBusNamespace, Set-AzServiceBusNamespace, Remove-AzServiceBusNamespace...} Script 3.0.1 Az.ServiceFabric {Add-AzServiceFabricClientCertificate, Add-AzServiceFabricNode, Add-AzServiceFabricNodeType, Get-AzServiceFabricClust... Script 1.3.0 Az.SignalR {New-AzSignalR, Get-AzSignalR, Get-AzSignalRKey, New-AzSignalRKey...} Script 3.5.0 Az.Sql {Get-AzSqlDatabaseTransparentDataEncryption, Get-AzSqlDatabaseTransparentDataEncryptionActivity, Set-AzSqlDatabaseTra... Script 1.1.0 Az.SqlVirtualMachine {New-AzSqlVM, Get-AzSqlVM, Update-AzSqlVM, Remove-AzSqlVM...} Script 3.11.0 Az.Storage {Get-AzStorageAccount, Get-AzStorageAccountKey, New-AzStorageAccount, New-AzStorageAccountKey...} Script 1.6.0 Az.StorageSync {Invoke-AzStorageSyncCompatibilityCheck, New-AzStorageSyncService, Get-AzStorageSyncService, Set-AzStorageSyncService... Script 2.0.0 Az.StreamAnalytics {Get-AzStreamAnalyticsCluster, Get-AzStreamAnalyticsClusterStreamingJob, Get-AzStreamAnalyticsDefaultFunctionDefiniti... Script 1.0.0 Az.Support {Get-AzSupportService, Get-AzSupportProblemClassification, Get-AzSupportTicket, Get-AzSupportTicketCommunication...} Script 1.0.4 Az.TrafficManager {Add-AzTrafficManagerCustomHeaderToEndpoint, Remove-AzTrafficManagerCustomHeaderFromEndpoint, Add-AzTrafficManagerCus... Script 2.8.2 Az.Websites {Get-AzAppServicePlan, Set-AzAppServicePlan, New-AzAppServicePlan, Remove-AzAppServicePlan...} Script 1.1.20 dbatools {Select-DbaObject, Set-DbatoolsConfig, Add-DbaAgDatabase, Add-DbaAgListener...} Script 1.0.1 Microsoft.PowerShell.Operation.V... {Get-OperationValidation, Invoke-OperationValidation} Script 5.3.1 oh-my-posh {Get-PoshThemes, Set-PoshPrompt, Get-PoshInfoForV2Users, Show-Colors...} Script 1.4.7 PackageManagement {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...} Binary 1.0.0.1 PackageManagement {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...} Script 3.4.0 Pester {Describe, Context, It, Should...} Script 1.0.0 posh-git {Add-PoshGitToProfile, Expand-GitCommand, Format-GitBranchName, Get-GitBranchStatusColor...} Script 2.2.5 PowerShellGet {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability...} Script 1.0.0.1 PowerShellGet {Install-Module, Find-Module, Save-Module, Update-Module...} Script 2.0.0 PSReadline {Get-PSReadLineKeyHandler, Set-PSReadLineKeyHandler, Remove-PSReadLineKeyHandler, Get-PSReadLineOption...} Script 21.1.18256 SqlServer {Add-RoleMember, Add-SqlAvailabilityDatabase, Add-SqlAvailabilityGroupListenerStaticIp, Add-SqlAzureAuthenticationCon... Directory: C:\Windows\system32\WindowsPowerShell\v1.0\Modules ModuleType Version Name ExportedCommands ---------- ------- ---- ---------------- Manifest 1.0.0.0 ActiveDirectory {Add-ADCentralAccessPolicyMember, Add-ADComputerServiceAccount, Add-ADDomainControllerPasswordReplicationPolicy, Add-... Manifest 1.0.0.0 AppBackgroundTask {Disable-AppBackgroundTaskDiagnosticLog, Enable-AppBackgroundTaskDiagnosticLog, Set-AppBackgroundTaskResourcePolicy, ... Manifest 2.0.0.0 AppLocker {Get-AppLockerFileInformation, Get-AppLockerPolicy, New-AppLockerPolicy, Set-AppLockerPolicy...} Manifest 1.0.0.0 AppvClient {Add-AppvClientConnectionGroup, Add-AppvClientPackage, Add-AppvPublishingServer, Disable-Appv...} Manifest 2.0.1.0 Appx {Add-AppxPackage, Get-AppxPackage, Get-AppxPackageManifest, Remove-AppxPackage...} Script 1.0.0.0 AssignedAccess {Clear-AssignedAccess, Get-AssignedAccess, Set-AssignedAccess} Manifest 1.0 BestPractices {Get-BpaModel, Get-BpaResult, Invoke-BpaModel, Set-BpaResult} Manifest 1.0.0.0 BitLocker {Unlock-BitLocker, Suspend-BitLocker, Resume-BitLocker, Remove-BitLockerKeyProtector...} Manifest 2.0.0.0 BitsTransfer {Add-BitsFile, Complete-BitsTransfer, Get-BitsTransfer, Remove-BitsTransfer...} Manifest 1.0.0.0 BranchCache {Add-BCDataCacheExtension, Clear-BCCache, Disable-BC, Disable-BCDowngrading...} Manifest 1.0.0.0 CimCmdlets {Get-CimAssociatedInstance, Get-CimClass, Get-CimInstance, Get-CimSession...} Binary 2.0.0.0 ClusterAwareUpdating {Get-CauPlugin, Register-CauPlugin, Unregister-CauPlugin, Invoke-CauScan...} Manifest 1.0 ConfigCI {Get-SystemDriver, New-CIPolicyRule, New-CIPolicy, Get-CIPolicy...} Manifest 1.0 ConfigDefender {Get-MpPreference, Set-MpPreference, Add-MpPreference, Remove-MpPreference...} Manifest 1.0 Defender {Get-MpPreference, Set-MpPreference, Add-MpPreference, Remove-MpPreference...} Manifest 1.0.2.0 DeliveryOptimization {Delete-DeliveryOptimizationCache, Set-DeliveryOptimizationStatus, Get-DeliveryOptimizationLog, Get-DeliveryOptimizat... Manifest 1.0 DFSN {Get-DfsnRoot, Remove-DfsnRoot, Set-DfsnRoot, New-DfsnRoot...} Binary 2.0.0.0 DFSR {New-DfsReplicationGroup, Get-DfsReplicationGroup, Set-DfsReplicationGroup, Remove-DfsReplicationGroup...} Manifest 2.0.0.0 DhcpServer {Add-DhcpServerInDC, Add-DhcpServerv4Class, Add-DhcpServerv4ExclusionRange, Add-DhcpServerv4Failover...} Manifest 1.0.0.0 DirectAccessClientComponents {Disable-DAManualEntryPointSelection, Enable-DAManualEntryPointSelection, Get-DAClientExperienceConfiguration, Get-DA... Script 3.0 Dism {Add-AppxProvisionedPackage, Add-WindowsDriver, Add-WindowsCapability, Add-WindowsImage...} Manifest 1.0.0.0 DnsClient {Resolve-DnsName, Clear-DnsClientCache, Get-DnsClient, Get-DnsClientCache...} Manifest 2.0.0.0 DnsServer {Add-DnsServerConditionalForwarderZone, Add-DnsServerDirectoryPartition, Add-DnsServerForwarder, Add-DnsServerPrimary... Manifest 1.0.0.0 EventTracingManagement {Start-EtwTraceSession, New-EtwTraceSession, Get-EtwTraceSession, Update-EtwTraceSession...} Manifest 2.0.0.0 FailoverClusters {Add-ClusterCheckpoint, Add-ClusterDisk, Add-ClusterFileServerRole, Add-ClusterGenericApplicationRole...} Manifest 1.0.0.0 GroupPolicy {Backup-GPO, Block-GPInheritance, Copy-GPO, Get-GPInheritance...} Manifest 1.0.0.0 HgsClient {Get-HgsAttestationBaselinePolicy, Get-HgsClientConfiguration, Test-HgsClientConfiguration, Set-HgsClientConfiguratio... Manifest 2.0.0.0 International {Get-WinDefaultInputMethodOverride, Set-WinDefaultInputMethodOverride, Get-WinHomeLocation, Set-WinHomeLocation...} Manifest 2.0.0.0 IpamServer {Get-IpamDhcpConfigurationEvent, Remove-IpamDhcpConfigurationEvent, Get-IpamConfigurationEvent, Remove-IpamConfigurat... Manifest 1.0.0.0 iSCSI {Get-IscsiTargetPortal, New-IscsiTargetPortal, Remove-IscsiTargetPortal, Update-IscsiTargetPortal...} Manifest 2.0.0.0 IscsiTarget {Add-ClusteriSCSITargetServerRole, Add-IscsiVirtualDiskTargetMapping, Checkpoint-IscsiVirtualDisk, Convert-IscsiVirtu... Script 1.0.0.0 ISE {New-IseSnippet, Import-IseSnippet, Get-IseSnippet} Manifest 1.0.0.0 Kds {Add-KdsRootKey, Get-KdsRootKey, Test-KdsRootKey, Set-KdsConfiguration...} Manifest 1.0.1.0 Microsoft.PowerShell.Archive {Compress-Archive, Expand-Archive} Manifest 3.0.0.0 Microsoft.PowerShell.Diagnostics {Get-WinEvent, Get-Counter, Import-Counter, Export-Counter...} Manifest 3.0.0.0 Microsoft.PowerShell.Host {Start-Transcript, Stop-Transcript} Manifest 1.0.0.0 Microsoft.PowerShell.LocalAccounts {Add-LocalGroupMember, Disable-LocalUser, Enable-LocalUser, Get-LocalGroup...} Manifest 3.1.0.0 Microsoft.PowerShell.Management {Add-Content, Clear-Content, Clear-ItemProperty, Join-Path...} Script 1.0 Microsoft.PowerShell.ODataUtils Export-ODataEndpointProxy Manifest 3.0.0.0 Microsoft.PowerShell.Security {Get-Acl, Set-Acl, Get-PfxCertificate, Get-Credential...} Manifest 3.1.0.0 Microsoft.PowerShell.Utility {Format-List, Format-Custom, Format-Table, Format-Wide...} Manifest 3.0.0.0 Microsoft.WSMan.Management {Disable-WSManCredSSP, Enable-WSManCredSSP, Get-WSManCredSSP, Set-WSManQuickConfig...} Manifest 1.0 MMAgent {Disable-MMAgent, Enable-MMAgent, Set-MMAgent, Get-MMAgent...} Manifest 1.0.0.0 MsDtc {New-DtcDiagnosticTransaction, Complete-DtcDiagnosticTransaction, Join-DtcDiagnosticResourceManager, Receive-DtcDiagn... Manifest 2.0.0.0 NetAdapter {Disable-NetAdapter, Disable-NetAdapterBinding, Disable-NetAdapterChecksumOffload, Disable-NetAdapterEncapsulatedPack... Manifest 1.0.0.0 NetConnection {Get-NetConnectionProfile, Set-NetConnectionProfile} Manifest 1.0.0.0 NetEventPacketCapture {New-NetEventSession, Remove-NetEventSession, Get-NetEventSession, Set-NetEventSession...} Manifest 2.0.0.0 NetLbfo {Add-NetLbfoTeamMember, Add-NetLbfoTeamNic, Get-NetLbfoTeam, Get-NetLbfoTeamMember...} Manifest 1.0 NetLldpAgent {Enable-NetLldpAgent, Disable-NetLldpAgent, Get-NetLldpAgent} Manifest 1.0.0.0 NetNat {Get-NetNat, Get-NetNatExternalAddress, Get-NetNatStaticMapping, Get-NetNatSession...} Manifest 2.0.0.0 NetQos {Get-NetQosPolicy, Set-NetQosPolicy, Remove-NetQosPolicy, New-NetQosPolicy} Manifest 2.0.0.0 NetSecurity {Get-DAPolicyChange, New-NetIPsecAuthProposal, New-NetIPsecMainModeCryptoProposal, New-NetIPsecQuickModeCryptoProposa... Manifest 1.0.0.0 NetSwitchTeam {New-NetSwitchTeam, Remove-NetSwitchTeam, Get-NetSwitchTeam, Rename-NetSwitchTeam...} Manifest 1.0.0.0 NetTCPIP {Get-NetIPAddress, Get-NetIPInterface, Get-NetIPv4Protocol, Get-NetIPv6Protocol...} Manifest 1.0.0.0 NetworkConnectivityStatus {Get-DAConnectionStatus, Get-NCSIPolicyConfiguration, Reset-NCSIPolicyConfiguration, Set-NCSIPolicyConfiguration} Manifest 1.0.0.0 NetworkController {Add-NetworkControllerNode, Disable-NetworkControllerNode, Enable-NetworkControllerNode, Get-NetworkControllerCluster... Manifest 1.0.0.0 NetworkControllerDiagnostics {Get-NetworkControllerDeploymentInfo, Get-NetworkControllerManagedDevices, Debug-NetworkController, Debug-NetworkCont... Manifest 2.0.0.0 NetworkLoadBalancingClusters {Add-NlbClusterNode, Add-NlbClusterNodeDip, Add-NlbClusterPortRule, Add-NlbClusterVip...} Manifest 1.0.0.0 NetworkSwitchManager {Disable-NetworkSwitchEthernetPort, Enable-NetworkSwitchEthernetPort, Get-NetworkSwitchEthernetPort, Remove-NetworkSw... Manifest 1.0.0.0 NetworkTransition {Add-NetIPHttpsCertBinding, Disable-NetDnsTransitionConfiguration, Disable-NetIPHttpsProfile, Disable-NetNatTransitio... Manifest 1.0 NFS {Get-NfsMappedIdentity, Get-NfsNetgroup, Install-NfsMappingStore, New-NfsMappedIdentity...} Manifest 1.0.0.0 PcsvDevice {Get-PcsvDevice, Start-PcsvDevice, Stop-PcsvDevice, Restart-PcsvDevice...} Binary 1.0.0.0 PersistentMemory {Get-PmemDisk, Get-PmemPhysicalDevice, Get-PmemUnusedRegion, New-PmemDisk...} Manifest 1.0.0.0 PKI {Add-CertificateEnrollmentPolicyServer, Export-Certificate, Export-PfxCertificate, Get-CertificateAutoEnrollmentPolic... Manifest 1.0.0.0 PnpDevice {Get-PnpDevice, Get-PnpDeviceProperty, Enable-PnpDevice, Disable-PnpDevice} Manifest 1.1 PrintManagement {Add-Printer, Add-PrinterDriver, Add-PrinterPort, Get-PrintConfiguration...} Binary 1.0.12 ProcessMitigations {Get-ProcessMitigation, Set-ProcessMitigation, ConvertTo-ProcessMitigationPolicy} Script 3.0 Provisioning {Install-ProvisioningPackage, Export-ProvisioningPackage, Install-TrustedProvisioningCertificate, Export-Trace...} Manifest 1.1 PSDesiredStateConfiguration {Set-DscLocalConfigurationManager, Start-DscConfiguration, Test-DscConfiguration, Publish-DscConfiguration...} Script 1.0.0.0 PSDiagnostics {Disable-PSTrace, Disable-PSWSManCombinedTrace, Disable-WSManTrace, Enable-PSTrace...} Binary 1.1.0.0 PSScheduledJob {New-JobTrigger, Add-JobTrigger, Remove-JobTrigger, Get-JobTrigger...} Manifest 2.0.0.0 PSWorkflow {New-PSWorkflowExecutionOption, New-PSWorkflowSession, nwsn} Manifest 1.0.0.0 PSWorkflowUtility Invoke-AsWorkflow Manifest 3.0.0.0 RemoteAccess {Add-DAAppServer, Add-DAClient, Add-DAClientDnsConfiguration, Add-DAEntryPoint...} Manifest 2.0.0.0 RemoteDesktop {Get-RDCertificate, Set-RDCertificate, New-RDCertificate, New-RDVirtualDesktopDeployment...} Manifest 1.0.0.0 ScheduledTasks {Get-ScheduledTask, Set-ScheduledTask, Register-ScheduledTask, Unregister-ScheduledTask...} Manifest 2.0.0.0 SecureBoot {Confirm-SecureBootUEFI, Set-SecureBootUEFI, Get-SecureBootUEFI, Format-SecureBootUEFI...} Script 2.0.0.0 ServerManager {Get-WindowsFeature, Install-WindowsFeature, Uninstall-WindowsFeature, Enable-ServerManagerStandardUserRemoting...} Cim 1.0.0.0 ServerManagerTasks {Get-SMCounterSample, Get-SMPerformanceCollector, Start-SMPerformanceCollector, Stop-SMPerformanceCollector...} Manifest 1.0.0.0 ShieldedVMDataFile {Import-ShieldingDataFile, New-ShieldingDataFile, New-VolumeIDQualifier, Save-VolumeSignatureCatalog...} Manifest 1.0.0.0 ShieldedVMTemplate {Protect-TemplateDisk, Initialize-VMShieldingHelperVHD} Manifest 2.0.0.0 SmbShare {Get-SmbShare, Remove-SmbShare, Set-SmbShare, Block-SmbShareAccess...} Manifest 2.0.0.0 SmbWitness {Get-SmbWitnessClient, Move-SmbWitnessClient, gsmbw, msmbw...} Manifest 1.0.0.1 StartLayout {Export-StartLayout, Import-StartLayout, Export-StartLayoutEdgeAssets, Get-StartApps} Manifest 2.0.0.0 Storage {Add-InitiatorIdToMaskingSet, Add-PartitionAccessPath, Add-PhysicalDisk, Add-StorageFaultDomain...} Manifest 1.0.0.0 StorageBusCache {Clear-StorageBusDisk, Disable-StorageBusCache, Disable-StorageBusDisk, Enable-StorageBusCache...} Manifest 1.0.0.0 StorageQoS {Get-StorageQoSPolicy, Get-StorageQoSPolicyStore, Set-StorageQoSPolicyStore, Remove-StorageQoSPolicy...} Manifest 1.0 StorageReplica {Test-SRTopology, New-SRGroup, Remove-SRGroup, Set-SRGroup...} Manifest 2.0.0.0 TLS {New-TlsSessionTicketKey, Enable-TlsSessionTicketKey, Disable-TlsSessionTicketKey, Export-TlsSessionTicketKey...} Manifest 1.0.0.0 TroubleshootingPack {Get-TroubleshootingPack, Invoke-TroubleshootingPack} Manifest 2.0.0.0 TrustedPlatformModule {Get-Tpm, Initialize-Tpm, Clear-Tpm, Unblock-Tpm...} Binary 2.1.639.0 UEV {Clear-UevConfiguration, Clear-UevAppxPackage, Restore-UevBackup, Set-UevTemplateProfile...} Manifest 2.0.0.0 UpdateServices {Add-WsusComputer, Approve-WsusUpdate, Deny-WsusUpdate, Get-WsusClassification...} Manifest 2.0.0.0 VpnClient {Add-VpnConnection, Set-VpnConnection, Remove-VpnConnection, Get-VpnConnection...} Manifest 1.0.0.0 Wdac {Get-OdbcDriver, Set-OdbcDriver, Get-OdbcDsn, Add-OdbcDsn...} Manifest 2.0.0.0 Whea {Get-WheaMemoryPolicy, Set-WheaMemoryPolicy} Manifest 1.0.0.0 WindowsDeveloperLicense {Get-WindowsDeveloperLicense, Unregister-WindowsDeveloperLicense, Show-WindowsDeveloperLicenseRegistration} Script 1.0 WindowsErrorReporting {Enable-WindowsErrorReporting, Disable-WindowsErrorReporting, Get-WindowsErrorReporting} Manifest 1.0.0.0 WindowsSearch {Get-WindowsSearchSetting, Set-WindowsSearchSetting} Manifest 1.0.0.0 WindowsUpdate Get-WindowsUpdateLog Directory: C:\Users\GAS02047\.vscode-insiders\extensions\ms-vscode.powershell-2021.9.0\modules ModuleType Version Name ExportedCommands ---------- ------- ---- ---------------- Script 1.1.3 Plaster {Invoke-Plaster, New-PlasterManifest, Get-PlasterTemplate, Test-PlasterManifest} Binary 2.5.1 PowerShellEditorServices Start-EditorServices Binary 0.2.0 PowerShellEditorServices.VSCode {New-VSCodeHtmlContentView, Show-VSCodeHtmlContentView, Close-VSCodeHtmlContentView, Set-VSCodeHtmlContentView...} Script 2.1.0 PSReadLine {Get-PSReadLineKeyHandler, Set-PSReadLineKeyHandler, Remove-PSReadLineKeyHandler, Get-PSReadLineOption...} Script 1.20.0 PSScriptAnalyzer {Get-ScriptAnalyzerRule, Invoke-ScriptAnalyzer, Invoke-Formatter} ``` ## Debug output Output when it works from the point where I output the contents of the $Confg variable: ``` { "TargetServer": "dwqa.gassouth.com", "TargetInstance": "MSSQLSERVER", "RestoredDBName": "GSDWCON", "TargetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA", "OverwriteWLIfpresent": "Yes", "NoRecoveryMode": "Disabled", "targetPhysicalPath": [ { "MappingType": "Data", "SourceLogicalName": "GSDWCON", "SourcePath": "R:\\Data\\GSDWCON.mdf", "TargetPath": "E:\\SQLData\\GSDWCON\\GSDWCON.mdf" }, { "MappingType": "Log", "SourceLogicalName": "GSDWCON_log", "SourcePath": "Z:\\Log\\GSDWCON_log.ldf", "TargetPath": "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf" } ], "ContainerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa", "FullRP": null, "SourceResourceId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01", "RestoreRequestType": "Alternate WL Restore to diff item", "RecoveryPoint": { "RecoveryPointTier": 0, "RecoveryPointMoveReadinessInfo": null, "RehydrationExpiryTime": null, "DataDirectoryPaths": null, "RecoveryPointType": null, "RecoveryPointTime": "\/Date(-62135578800000)\/", "RecoveryPointId": "DefaultRangeRecoveryPoint", "ItemName": null, "Id": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint", "WorkloadType": 0, "ContainerName": null, "ContainerType": 0, "BackupManagementType": 0 }, "PointInTime": "\/Date(1632756730000)\/", "RecoveryMode": null, "FilePath": null } DEBUG: 1:59:04 PM - RestoreAzureRmRecoveryServicesBackupItem begin processing with ParameterSet 'AzureWorkloadParameterSet'. DEBUG: 1:59:04 PM - using account id 'GAS02047@GasSouth.com'... DEBUG: Inside Restore. Going to create ResourceClient. DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec' DEBUG: 1:59:04 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com' DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Found 1 cache accounts and 0 broker accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Returning 1 accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(63d69379-1e96-4b55-abe5-f46777b338c0) DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === AcquireTokenSilent Parameters === DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] LoginHint provided: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Account provided: True DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] ForceRefresh: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Request Data === Authority Provided? - True Scopes - https://management.core.windows.net//.default Extra Query Params Keys (space separated) - ApiId - AcquireTokenSilent IsConfidentialClient - False SendX5C - False LoginHint ? False IsBrokerConfigured - False HomeAccountId - False CorrelationId - 63d69379-1e96-4b55-abe5-f46777b338c0 DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition (SilentRequest) started: Authority Host: login.microsoftonline.com DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 17:59:04) - Expiration Time (09/27/2021 18:33:33 +00:00) - Extended Expiration Time (09/27/2021 18:33:33 +00:00)] DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Returning access token found in cache. RefreshOn exists ? False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Fetched access token from host login.microsoftonline.com. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition finished successfully. An access token was returned with Expiration Time: 09/27/2021 18:33:33 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: ExpiresOn: 2021-09-27T18:33:33.0000000+00:00 DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com' DEBUG: Client Created successfully DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec' DEBUG: 1:59:04 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com' DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Found 1 cache accounts and 0 broker accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04] Returning 1 accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(8fbbef67-e99e-43a2-83b8-87f3ad3e0290) DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === AcquireTokenSilent Parameters === DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] LoginHint provided: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Account provided: True DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] ForceRefresh: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Request Data === Authority Provided? - True Scopes - https://management.core.windows.net//.default Extra Query Params Keys (space separated) - ApiId - AcquireTokenSilent IsConfidentialClient - False SendX5C - False LoginHint ? False IsBrokerConfigured - False HomeAccountId - False CorrelationId - 8fbbef67-e99e-43a2-83b8-87f3ad3e0290 DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition (SilentRequest) started: Authority Host: login.microsoftonline.com DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 17:59:04) - Expiration Time (09/27/2021 18:33:33 +00:00) - Extended Expiration Time (09/27/2021 18:33:33 +00:00)] DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Returning access token found in cache. RefreshOn exists ? False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] Fetched access token from host login.microsoftonline.com. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 17:59:04 - ] === Token Acquisition finished successfully. An access token was returned with Expiration Time: 09/27/2021 18:33:33 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: ExpiresOn: 2021-09-27T18:33:33.0000000+00:00 DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com' DEBUG: ============================ HTTP REQUEST ============================ HTTP Method: POST Absolute Uri: https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint/restore?api-version=2021-06-01 Headers: x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d accept-language : en-US Body: { "properties": { "objectType": "AzureWorkloadSQLPointInTimeRestoreRequest", "pointInTime": "2021-09-27T15:32:10Z", "shouldUseAlternateTargetLocation": true, "isNonRecoverable": false, "alternateDirectoryPaths": [ { "mappingType": "Data", "sourceLogicalName": "GSDWCON", "sourcePath": "R:\\Data\\GSDWCON.mdf", "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON.mdf" }, { "mappingType": "Log", "sourceLogicalName": "GSDWCON_log", "sourcePath": "Z:\\Log\\GSDWCON_log.ldf", "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf" } ], "recoveryType": "AlternateLocation", "sourceResourceId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01", "targetInfo": { "overwriteOption": "Overwrite", "containerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa", "databaseName": "MSSQLSERVER/GSDWCON" }, "targetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA" } } DEBUG: ============================ HTTP RESPONSE ============================ Status Code: Accepted Headers: Pragma : no-cache Retry-After : 60 Azure-AsyncOperation : https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 X-Content-Type-Options : nosniff x-ms-request-id : 6e03b616-cb10-43a0-a781-e5301876ea15 x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d Strict-Transport-Security : max-age=31536000; includeSubDomains x-ms-ratelimit-remaining-subscription-writes: 1199 x-ms-correlation-request-id : 6e03b616-cb10-43a0-a781-e5301876ea15 x-ms-routing-request-id : CENTRALUS:20210927T175906Z:6e03b616-cb10-43a0-a781-e5301876ea15 Cache-Control : no-cache Date : Mon, 27 Sep 2021 17:59:05 GMT Location : https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationResults/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 X-Powered-By : ASP.NET Body: DEBUG: ============================ HTTP REQUEST ============================ HTTP Method: GET Absolute Uri: https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 Headers: x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d Body: DEBUG: ============================ HTTP RESPONSE ============================ Status Code: OK Headers: Pragma : no-cache X-Content-Type-Options : nosniff x-ms-request-id : 665a97ef-af5d-45ac-bf77-6875ecd27aec x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d Strict-Transport-Security : max-age=31536000; includeSubDomains x-ms-ratelimit-remaining-subscription-resource-requests: 149 x-ms-correlation-request-id : 665a97ef-af5d-45ac-bf77-6875ecd27aec x-ms-routing-request-id : CENTRALUS:20210927T180006Z:665a97ef-af5d-45ac-bf77-6875ecd27aec Cache-Control : no-cache Date : Mon, 27 Sep 2021 18:00:06 GMT Server : Microsoft-IIS/10.0 X-Powered-By : ASP.NET Body: { "id": "dc370263-0e68-4ed9-b266-5a84c6731e06", "name": "dc370263-0e68-4ed9-b266-5a84c6731e06", "status": "Succeeded", "startTime": "2021-09-27T17:59:05.7299304Z", "endTime": "2021-09-27T17:59:05.7299304Z", "properties": { "objectType": "OperationStatusJobExtendedInfo", "jobId": "dc370263-0e68-4ed9-b266-5a84c6731e06" } } DEBUG: ============================ HTTP REQUEST ============================ HTTP Method: GET Absolute Uri: https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationResults/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 Headers: x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d Body: DEBUG: ============================ HTTP RESPONSE ============================ Status Code: NoContent Headers: Pragma : no-cache Retry-After : 60 Azure-AsyncOperation : https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 X-Content-Type-Options : nosniff x-ms-request-id : 95621390-59e5-4c14-99d9-b1107af1bf49 x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d Strict-Transport-Security : max-age=31536000; includeSubDomains x-ms-ratelimit-remaining-subscription-resource-requests: 149 x-ms-correlation-request-id : 95621390-59e5-4c14-99d9-b1107af1bf49 x-ms-routing-request-id : CENTRALUS:20210927T180007Z:95621390-59e5-4c14-99d9-b1107af1bf49 Cache-Control : no-cache Date : Mon, 27 Sep 2021 18:00:07 GMT X-Powered-By : ASP.NET Body: DEBUG: Tracking operation status URL for completion:https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/operationsStatus/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 DEBUG: ============================ HTTP REQUEST ============================ HTTP Method: GET Absolute Uri: https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupOperations/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 Headers: x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d accept-language : en-US Body: DEBUG: ============================ HTTP RESPONSE ============================ Status Code: OK Headers: Pragma : no-cache X-Content-Type-Options : nosniff x-ms-request-id : d0091d1c-cc84-4a54-90fc-57e36fe56279 x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d Strict-Transport-Security : max-age=31536000; includeSubDomains x-ms-ratelimit-remaining-subscription-resource-requests: 149 x-ms-correlation-request-id : d0091d1c-cc84-4a54-90fc-57e36fe56279 x-ms-routing-request-id : CENTRALUS:20210927T180008Z:d0091d1c-cc84-4a54-90fc-57e36fe56279 Cache-Control : no-cache Date : Mon, 27 Sep 2021 18:00:07 GMT Server : Microsoft-IIS/10.0 X-Powered-By : ASP.NET Body: { "id": "dc370263-0e68-4ed9-b266-5a84c6731e06", "name": "dc370263-0e68-4ed9-b266-5a84c6731e06", "status": "Succeeded", "startTime": "2021-09-27T17:59:05.7299304Z", "endTime": "2021-09-27T17:59:05.7299304Z", "properties": { "objectType": "OperationStatusJobExtendedInfo", "jobId": "dc370263-0e68-4ed9-b266-5a84c6731e06" } } DEBUG: ============================ HTTP REQUEST ============================ HTTP Method: GET Absolute Uri: https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupOperations/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 Headers: x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d accept-language : en-US Body: DEBUG: ============================ HTTP RESPONSE ============================ Status Code: OK Headers: Pragma : no-cache X-Content-Type-Options : nosniff x-ms-request-id : 37362667-662b-40f5-aa0d-c9e1e49b61e2 x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d Strict-Transport-Security : max-age=31536000; includeSubDomains x-ms-ratelimit-remaining-subscription-resource-requests: 148 x-ms-correlation-request-id : 37362667-662b-40f5-aa0d-c9e1e49b61e2 x-ms-routing-request-id : CENTRALUS:20210927T180008Z:37362667-662b-40f5-aa0d-c9e1e49b61e2 Cache-Control : no-cache Date : Mon, 27 Sep 2021 18:00:08 GMT Server : Microsoft-IIS/10.0 X-Powered-By : ASP.NET Body: { "id": "dc370263-0e68-4ed9-b266-5a84c6731e06", "name": "dc370263-0e68-4ed9-b266-5a84c6731e06", "status": "Succeeded", "startTime": "2021-09-27T17:59:05.7299304Z", "endTime": "2021-09-27T17:59:05.7299304Z", "properties": { "objectType": "OperationStatusJobExtendedInfo", "jobId": "dc370263-0e68-4ed9-b266-5a84c6731e06" } } DEBUG: Final operation status:Succeeded DEBUG: ============================ HTTP REQUEST ============================ HTTP Method: GET Absolute Uri: https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupJobs/dc370263-0e68-4ed9-b266-5a84c6731e06?api-version=2021-06-01 Headers: x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d accept-language : en-US Body: DEBUG: ============================ HTTP RESPONSE ============================ Status Code: OK Headers: Pragma : no-cache Cache-Control : no-cache Server : Microsoft-IIS/10.0,Microsoft-IIS/10.0 X-Content-Type-Options : nosniff x-ms-request-id : 35c9bfcb-d282-4fb1-a833-bf5f99106748 x-ms-client-request-id : 77b77826-02eb-438c-a92f-84a78c570a3d,77b77826-02eb-438c-a92f-84a78c570a3d X-Powered-By : ASP.NET Strict-Transport-Security : max-age=31536000; includeSubDomains x-ms-ratelimit-remaining-subscription-resource-requests: 149 x-ms-correlation-request-id : 35c9bfcb-d282-4fb1-a833-bf5f99106748 x-ms-routing-request-id : CENTRALUS:20210927T180008Z:35c9bfcb-d282-4fb1-a833-bf5f99106748 Date : Mon, 27 Sep 2021 18:00:08 GMT Body: { "id": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupJobs/dc370263-0e68-4ed9-b266-5a84c6731e06", "name": "dc370263-0e68-4ed9-b266-5a84c6731e06", "type": "Microsoft.RecoveryServices/vaults/backupJobs", "properties": { "jobType": "AzureWorkloadJob", "actionsInfo": [ 1 ], "workloadType": "SQLDataBase", "duration": "PT1M3.0147601S", "extendedInfo": { "tasksList": [ { "taskId": "Transfer data from vault", "status": "InProgress" } ], "propertyBag": { "Job Type": "Recovery to an alternate database" } }, "isUserTriggered": true, "entityFriendlyName": "GSDWCON [dwprod01.gassouth.com]", "backupManagementType": "AzureWorkload", "operation": "Restore", "status": "InProgress", "startTime": "2021-09-27T17:59:05.7299304Z", "activityId": "77b77826-02eb-438c-a92f-84a78c570a3d" } } WorkloadName Operation Status StartTime EndTime JobID ------------ --------- ------ --------- ------- ----- GSDWCON [dwpr... Restore InProgress 9/27/2021 5:59:05 PM dc370263-0e68-4ed9-b266-5a84c6731e06 DEBUG: AzureQoSEvent: Module: Az.RecoveryServices:4.6.0; CommandName: Restore-AzRecoveryServicesBackupItem; PSVersion: 5.1.19041.1237; IsSuccess: True; Duration: 00:01:04.2375683 DEBUG: Finish sending metric. DEBUG: 2:00:09 PM - RestoreAzureRmRecoveryServicesBackupItem end processing. ``` Output when it fails: ``` { "TargetServer": "dwqa.gassouth.com", "TargetInstance": "MSSQLSERVER", "RestoredDBName": "GSDWCON", "TargetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA", "OverwriteWLIfpresent": "Yes", "NoRecoveryMode": "Disabled", "targetPhysicalPath": [ { "MappingType": "Data", "SourceLogicalName": "GSDWCON", "SourcePath": "R:\\Data\\GSDWCON.mdf", "TargetPath": "E:\\SQLData\\GSDWCON\\GSDWCON.mdf" }, { "MappingType": "Log", "SourceLogicalName": "GSDWCON_log", "SourcePath": "Z:\\Log\\GSDWCON_log.ldf", "TargetPath": "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf" } ], "ContainerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa", "FullRP": null, "SourceResourceId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01", "RestoreRequestType": "Alternate WL Restore to diff item", "RecoveryPoint": { "RecoveryPointTier": 0, "RecoveryPointMoveReadinessInfo": null, "RehydrationExpiryTime": null, "DataDirectoryPaths": null, "RecoveryPointType": null, "RecoveryPointTime": "\/Date(-62135578800000)\/", "RecoveryPointId": "DefaultRangeRecoveryPoint", "ItemName": null, "Id": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint", "WorkloadType": 0, "ContainerName": null, "ContainerType": 0, "BackupManagementType": 0 }, "PointInTime": "\/Date(1632771248000)\/", "RecoveryMode": null, "FilePath": null } DEBUG: 3:42:25 PM - RestoreAzureRmRecoveryServicesBackupItem begin processing with ParameterSet 'AzureWorkloadParameterSet'. DEBUG: 3:42:25 PM - using account id 'GAS02047@GasSouth.com'... DEBUG: Inside Restore. Going to create ResourceClient. DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec' DEBUG: 3:42:25 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com' DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Found 1 cache accounts and 0 broker accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Returning 1 accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(e0eb4966-afbb-465a-9b60-cf231b31fddb) DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === AcquireTokenSilent Parameters === DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] LoginHint provided: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Account provided: True DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] ForceRefresh: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Request Data === Authority Provided? - True Scopes - https://management.core.windows.net//.default Extra Query Params Keys (space separated) - ApiId - AcquireTokenSilent IsConfidentialClient - False SendX5C - False LoginHint ? False IsBrokerConfigured - False HomeAccountId - False CorrelationId - e0eb4966-afbb-465a-9b60-cf231b31fddb DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition (SilentRequest) started: Authority Host: login.microsoftonline.com DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 19:42:25) - Expiration Time (09/27/2021 20:12:03 +00:00) - Extended Expiration Time (09/27/2021 20:12:03 +00:00)] DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Returning access token found in cache. RefreshOn exists ? False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Fetched access token from host login.microsoftonline.com. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition finished successfully. An access token was returned with Expiration Time: 09/27/2021 20:12:03 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: ExpiresOn: 2021-09-27T20:12:03.0000000+00:00 DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com' DEBUG: Client Created successfully DEBUG: [Common.Authentication]: Authenticating using Account: 'GAS02047@GasSouth.com', environment: 'AzureCloud', tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec' DEBUG: 3:42:25 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'ff0b5dfd-2901-4555-a244-19da12ce34ec', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'GAS02047@GasSouth.com' DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Found 1 cache accounts and 0 broker accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25] Returning 1 accounts DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] MSAL MSAL.Desktop with assembly version '4.30.1.0'. CorrelationId(3ad726ae-cf5b-4c45-ae8b-8363ad62ecbd) DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === AcquireTokenSilent Parameters === DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] LoginHint provided: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Account provided: True DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] ForceRefresh: False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Request Data === Authority Provided? - True Scopes - https://management.core.windows.net//.default Extra Query Params Keys (space separated) - ApiId - AcquireTokenSilent IsConfidentialClient - False SendX5C - False LoginHint ? False IsBrokerConfigured - False HomeAccountId - False CorrelationId - 3ad726ae-cf5b-4c45-ae8b-8363ad62ecbd DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition (SilentRequest) started: Authority Host: login.microsoftonline.com DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Azure region was not configured or could not be discovered. Not using a regional authority. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Access token is not expired. Returning the found cache entry. [Current time (09/27/2021 19:42:25) - Expiration Time (09/27/2021 20:12:03 +00:00) - Extended Expiration Time (09/27/2021 20:12:03 +00:00)] DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Returning access token found in cache. RefreshOn exists ? False DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] Fetched access token from host login.microsoftonline.com. DEBUG: (False) MSAL 4.30.1.0 MSAL.Desktop Windows 10 Enterprise [09/27/2021 19:42:25 - ] === Token Acquisition finished successfully. An access token was returned with Expiration Time: 09/27/2021 20:12:03 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: ExpiresOn: 2021-09-27T20:12:03.0000000+00:00 DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'ff0b5dfd-2901-4555-a244-19da12ce34ec', UserId: 'GAS02047@GasSouth.com' DEBUG: ============================ HTTP REQUEST ============================ HTTP Method: POST Absolute Uri: https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/recoveryPoints/DefaultRangeRecoveryPoint/restore?api-version=2021-06-01 Headers: x-ms-client-request-id : 36b8096b-73f3-4c91-a567-df2948237d2e accept-language : en-US Body: { "properties": { "objectType": "AzureWorkloadSQLPointInTimeRestoreRequest", "pointInTime": "2021-09-27T19:34:08Z", "shouldUseAlternateTargetLocation": true, "isNonRecoverable": false, "alternateDirectoryPaths": [ { "mappingType": "Data", "sourceLogicalName": "GSDWCON", "sourcePath": "R:\\Data\\GSDWCON.mdf", "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON.mdf" }, { "mappingType": "Log", "sourceLogicalName": "GSDWCON_log", "sourcePath": "Z:\\Log\\GSDWCON_log.ldf", "targetPath": "E:\\SQLData\\GSDWCON\\GSDWCON_log.ldf" } ], "recoveryType": "AlternateLocation", "sourceResourceId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/dwprod/providers/Microsoft.Compute/virtualMachines/dwprod01", "targetInfo": { "overwriteOption": "Overwrite", "containerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;compute;dwqa;dwqa", "databaseName": "MSSQLSERVER/GSDWCON" }, "targetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Microsoft.Compute/virtualMachines/DWQA" } } DEBUG: ============================ HTTP RESPONSE ============================ Status Code: BadRequest Headers: Pragma : no-cache X-Content-Type-Options : nosniff x-ms-request-id : 0eb7b15f-642b-4f94-bcd6-a63048b32a5c x-ms-client-request-id : 36b8096b-73f3-4c91-a567-df2948237d2e,36b8096b-73f3-4c91-a567-df2948237d2e Strict-Transport-Security : max-age=31536000; includeSubDomains x-ms-ratelimit-remaining-subscription-writes: 1199 x-ms-correlation-request-id : 0eb7b15f-642b-4f94-bcd6-a63048b32a5c x-ms-routing-request-id : CENTRALUS:20210927T194226Z:0eb7b15f-642b-4f94-bcd6-a63048b32a5c Cache-Control : no-cache Date : Mon, 27 Sep 2021 19:42:26 GMT Server : Microsoft-IIS/10.0 X-Powered-By : ASP.NET Body: { "error": { "code": "BMSUserErrorInvalidInput", "message": "Input provided for the call is invalid. Please check the required inputs", "target": null, "details": null, "innerError": null } } DEBUG: Caught exception, type: System.AggregateException DEBUG: Handling aggregate exception DEBUG: Received CloudException, ErrorCode: BMSUserErrorInvalidInput, Message: Input provided for the call is invalid. Please check the required inputs Restore-AzRecoveryServicesBackupItem : Input provided for the call is invalid. Please check the required inputs At C:\Users\GAS02047\OneDrive - Gas South\Documents\testcase-Restore-AzRecoveryServicesBackupItem.ps1:14 char:1 + Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [Restore-AzRecoveryServicesBackupItem], CloudException + FullyQualifiedErrorId : BMSUserErrorInvalidInput,Microsoft.Azure.Commands.RecoveryServices.Backup.Cmdlets.RestoreAzureRmRecoveryServicesBackupItem DEBUG: AzureQoSEvent: Module: Az.RecoveryServices:4.6.0; CommandName: Restore-AzRecoveryServicesBackupItem; PSVersion: 5.1.19041.1237; IsSuccess: False; Duration: 00:00:01.1917939; Exception: Input provided for the call is invalid. Please check the required inputs; DEBUG: Finish sending metric. DEBUG: 3:42:27 PM - RestoreAzureRmRecoveryServicesBackupItem end processing. ``` ## Error output ``` HistoryId: 18 RequestId : 0eb7b15f-642b-4f94-bcd6-a63048b32a5c Message : Input provided for the call is invalid. Please check the required inputs ServerMessage : BMSUserErrorInvalidInput: Input provided for the call is invalid. Please check the required inputs (System.Collections.Generic.List`1[Microsoft.Rest.Azure.CloudError]) ServerResponse : {BadRequest} RequestMessage : {POST https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/providers/Microsoft.RecoveryServices/ vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/VMAppContainer;compute;dwprod;dwprod01/protectedItems/SQLDataBase;mssqlserver;gsdwcon/reco veryPoints/DefaultRangeRecoveryPoint/restore?api-version=2021-06-01} InvocationInfo : {Restore-AzRecoveryServicesBackupItem} Line : Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config Position : At C:\Users\GAS02047\OneDrive - Gas South\Documents\testcase-Restore-AzRecoveryServicesBackupItem.ps1:14 char:1 + Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $Config + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ StackTrace : at Microsoft.Azure.Management.RecoveryServices.Backup.RestoresOperations.d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Management.RecoveryServices.Backup.RestoresOperations.d__5.MoveNext() HistoryId : 18 ```
Author: mwsutherland
Assignees: -
Labels: `Recovery Services Backup`, `Service Attention`, `question`, `customer-reported`, `needs-triage`
Milestone: -
mwsutherland commented 3 years ago

Tonight, I'm going to try a different work-around. I don't really need the latest log backup. The latest full will do. So, instead of using $BackupItem.LatestRecoveryPoint as the point in time, I'm going to use (Get-AzRecoveryServicesBackupRecoveryPoint -Item $BackupItem)[0].RecoveryPointTime.

pvrk commented 3 years ago

@mwsutherland : First of all that apologies that you have to go through all this effort to figure out the error. We should do a better job at telling you the exact error message other than just "InvalidInput". By any chance, do you have any debug of the failure where you reduced 5 mins from the latest recovery point?

Also, when you are trying to restore distinct recovery points, the parameter in the config generation should be "recoveryPoint" and NOT "pointInTime". Hence, please use (Get-AzRecoveryServicesBackupRecoveryPoint -Item $BackupItem)[0] as the input to -RecoveryPoint and don't use point-in-time

pvrk commented 3 years ago

@mwsutherland: Also I noticed that the JSON files for the final requests are identical for both outcomes. I mean the request which was successful which was made at 18:00 GMT is also the exact request made at 19:42 GMT which failed. I am surprised since as time passes, the last recovery point should be updated. Also if the exact same request is repeated at a later time, it should succeed.

Bottom line: Can you please provide debug of the latest failures after 5 min reduction to cross verify once more?

mwsutherland commented 3 years ago

@pvrk Last night's restore run turned out the same as the night before. Afterwards, I realized that it wasn't a test of what I thought because the full backup was also the most recent backup. I was thinking that a log backup would have happened in between, but I was forgetting how things were scheduled.

The script running at night to do "real" restores doesn't have debugging information turned on, so I don't have the same output for that as when I do my manual test restores of a much smaller database. I'll add the line to output that and see what I get. If no better options present themselves before the end of the say, I'll use 5 minutes before last recovery point.

I would like to do a -RecoveryPoint restore, but, unfortunately, that seems to be broken. I have a separate issue filed for that.

Unless I'm looking at the wrong JSON, I see two different point-in-times specified in the two requests. 2021-09-27T15:32:10Z (1632756730000) for the first and 2021-09-27T19:34:08Z (1632771248000) for the second.

mwsutherland commented 3 years ago

@pvrk It turns out I had $DebugPreference='Continue' set in the nightly job, but that part of the output wasn't being piped into the log file. Hopefully *> will take care of that and I'll have more output for those restore attempts tomorrow, If you have any ideas of things I can try with my test database, I can do that during the day.

mwsutherland commented 3 years ago

I didn't try any more experiments today, so all I have to add is the result of tonight's job run. Tonight, we got something new -- BOTH failed! This was using $PointInTime = ($BackupItem.LatestRecoveryPoint).AddMinutes(-5). I don't know if it is useful with both failing, but here is the debugging around the calls to Restore-AzRecoveryServicesBackupItem:

============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/p
roviders/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/SQLAGWorkLoa
dContainer;8ecb2090-546c-4a8b-8f0a-c35f8b6f4339/protectedItems/SQLDataBase;dwag;edw/recoveryPoints/DefaultRangeRecovery
Point/restore?api-version=2021-03-01

Headers:
x-ms-client-request-id        : 73ffd149-9868-45a1-bf67-bb3b4a47ecf6
accept-language               : en-US

Body:
{
  "properties": {
    "objectType": "AzureWorkloadSQLPointInTimeRestoreRequest",
    "pointInTime": "2021-09-29T20:59:38Z",
    "shouldUseAlternateTargetLocation": true,
    "isNonRecoverable": false,
    "alternateDirectoryPaths": [
      {
        "mappingType": "Data",
        "sourceLogicalName": "EDW",
        "sourcePath": "R:\\EDW_Data\\EDW.mdf",
        "targetPath": "E:\\SQLData\\EDW\\EDW.mdf"
      },
      {
        "mappingType": "Log",
        "sourceLogicalName": "EDW_log",
        "sourcePath": "Z:\\EDW_Logs\\EDW_log.ldf",
        "targetPath": "E:\\SQLData\\EDW\\EDW_log.ldf"
      },
      {
        "mappingType": "Log",
        "sourceLogicalName": "EDW2_Log",
        "sourcePath": "K:\\SSISDB\\EDW2_Log.ldf",
        "targetPath": "E:\\SQLData\\EDW\\EDW2_Log.ldf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "FG_Daily",
        "sourcePath": "R:\\Data\\FG_Daily.ndf",
        "targetPath": "E:\\SQLData\\EDW\\FG_Daily.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "FG_Month",
        "sourcePath": "R:\\Data\\FG_Month.ndf",
        "targetPath": "E:\\SQLData\\EDW\\FG_Month.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "FG_Yearly",
        "sourcePath": "R:\\Data\\FG_Yearly.ndf",
        "targetPath": "E:\\SQLData\\EDW\\FG_Yearly.ndf"
      }
    ],
    "recoveryType": "AlternateLocation",
    "targetInfo": {
      "overwriteOption": "Overwrite",
      "containerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/provide
rs/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;com
pute;dwqa;dwqa",
      "databaseName": "MSSQLSERVER/EDW"
    },
    "targetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Micros
oft.Compute/virtualMachines/DWQA"
  }
}

============================ HTTP RESPONSE ============================

Status Code:
BadRequest

Headers:
Pragma                        : no-cache
X-Content-Type-Options        : nosniff
x-ms-request-id               : 31d69319-3681-4b43-b554-30fc2fb20dc6
x-ms-client-request-id        : 73ffd149-9868-45a1-bf67-bb3b4a47ecf6,73ffd149-9868-45a1-bf67-bb3b4a47ecf6
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-writes: 1199
x-ms-correlation-request-id   : 31d69319-3681-4b43-b554-30fc2fb20dc6
x-ms-routing-request-id       : CENTRALUS:20210929T223029Z:31d69319-3681-4b43-b554-30fc2fb20dc6
Cache-Control                 : no-cache
Date                          : Wed, 29 Sep 2021 22:30:29 GMT
Server                        : Microsoft-IIS/10.0
X-Powered-By                  : ASP.NET

Body:
{
  "error": {
    "code": "BMSUserErrorInvalidInput",
    "message": "Input provided for the call is invalid. Please check the required inputs",
    "target": null,
    "details": null,
    "innerError": null
  }
}

Caught exception, type: System.AggregateException
Handling aggregate exception
Received CloudException, ErrorCode: BMSUserErrorInvalidInput, Message: Input provided for the call is invalid. Please 
check the required inputs
Restore-AzRecoveryServicesBackupItem : Input provided for the call is invalid. Please check the required inputs
At C:\scripts\RestoreDBFromVault.ps1:93 char:5
+     Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $AnotherIn ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Restore-AzRecoveryServicesBackupItem], CloudException
    + FullyQualifiedErrorId : BMSUserErrorInvalidInput,Microsoft.Azure.Commands.RecoveryServices.Backup.Cmdlets.Restor 
   eAzureRmRecoveryServicesBackupItem

AzureQoSEvent: Module: Az.RecoveryServices:4.4.0; CommandName: Restore-AzRecoveryServicesBackupItem; PSVersion: 
5.1.14409.1005; IsSuccess: False; Duration: 00:00:01.1330865; Exception: Input provided for the call is invalid. 
Please check the required inputs;
Finish sending metric.
6:30:30 PM - RestoreAzureRmRecoveryServicesBackupItem end processing.

and

============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
https://management.azure.com/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/p
roviders/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/SQLAGWorkLoa
dContainer;8ecb2090-546c-4a8b-8f0a-c35f8b6f4339/protectedItems/SQLDataBase;dwag;gsdwstg/recoveryPoints/DefaultRangeReco
veryPoint/restore?api-version=2021-03-01

Headers:
x-ms-client-request-id        : 9f05ec6a-07ea-4f2a-a938-384683b9dfe4
accept-language               : en-US

Body:
{
  "properties": {
    "objectType": "AzureWorkloadSQLPointInTimeRestoreRequest",
    "pointInTime": "2021-09-29T21:35:08Z",
    "shouldUseAlternateTargetLocation": true,
    "isNonRecoverable": false,
    "alternateDirectoryPaths": [
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_MDATA_FILE",
        "sourcePath": "R:\\detached_files\\Data\\DwStgMDatFil.mdf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgMDatFil.mdf"
      },
      {
        "mappingType": "Log",
        "sourceLogicalName": "DWSTG_LOG_FILE_01",
        "sourcePath": "Z:\\Log\\GSDWSTG_log.ldf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\GSDWSTG_log.ldf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_DATA_FILE_01",
        "sourcePath": "R:\\detached_files\\Data\\DwStgDatFil01.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgDatFil01.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_DATA_FILE_02",
        "sourcePath": "R:\\detached_files\\Data\\DwStgDatFil02.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgDatFil02.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_DATA_FILE_03",
        "sourcePath": "R:\\detached_files\\Data\\DwStgDatFil03.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgDatFil03.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_IDX_FILE_01",
        "sourcePath": "R:\\detached_files\\Data\\DwStgIdxFil01.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgIdxFil01.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_IDX_FILE_02",
        "sourcePath": "R:\\detached_files\\Data\\DwStgIdxFil02.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgIdxFil02.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_IDX_FILE_03",
        "sourcePath": "R:\\detached_files\\Data\\DwStgIdxFil03.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgIdxFil03.ndf"
      },
      {
        "mappingType": "Log",
        "sourceLogicalName": "DWSTG_LOG_FILE_02",
        "sourcePath": "Z:\\Log\\DWSTG_LOG_FILE_02.ldf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DWSTG_LOG_FILE_02.ldf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_LOG_FILE_04",
        "sourcePath": "R:\\detached_files\\Data\\DwStgDatFil04.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DwStgDatFil04.ndf"
      },
      {
        "mappingType": "Data",
        "sourceLogicalName": "DWSTG_MDATA_FILE2",
        "sourcePath": "Z:\\Log\\DWSTG_MDATA_FILE2.ndf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DWSTG_MDATA_FILE2.ndf"
      },
      {
        "mappingType": "Log",
        "sourceLogicalName": "DWSTG_LOG_FILE_03",
        "sourcePath": "K:\\GSDWSTG_log\\DWSTG_LOG_FILE_03.ldf",
        "targetPath": "E:\\SQLData\\GSDWSTG\\DWSTG_LOG_FILE_03.ldf"
      }
    ],
    "recoveryType": "AlternateLocation",
    "targetInfo": {
      "overwriteOption": "Overwrite",
      "containerId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/New-LRS-Recovery-Vault/provide
rs/Microsoft.RecoveryServices/vaults/New-LRS-Recovery-Vault/backupFabrics/Azure/protectionContainers/vmappcontainer;com
pute;dwqa;dwqa",
      "databaseName": "MSSQLSERVER/GSDWSTG"
    },
    "targetVirtualMachineId": "/subscriptions/13fc5eb3-f9be-4fda-84db-d051c04f9d50/resourceGroups/DWQA/providers/Micros
oft.Compute/virtualMachines/DWQA"
  }
}

============================ HTTP RESPONSE ============================

Status Code:
BadRequest

Headers:
Pragma                        : no-cache
X-Content-Type-Options        : nosniff
x-ms-request-id               : abaa36b3-059c-4aaf-8bfa-7b29b88cc8d3
x-ms-client-request-id        : 9f05ec6a-07ea-4f2a-a938-384683b9dfe4,9f05ec6a-07ea-4f2a-a938-384683b9dfe4
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-writes: 1199
x-ms-correlation-request-id   : abaa36b3-059c-4aaf-8bfa-7b29b88cc8d3
x-ms-routing-request-id       : CENTRALUS:20210929T223038Z:abaa36b3-059c-4aaf-8bfa-7b29b88cc8d3
Cache-Control                 : no-cache
Date                          : Wed, 29 Sep 2021 22:30:38 GMT
Server                        : Microsoft-IIS/10.0
X-Powered-By                  : ASP.NET

Body:
{
  "error": {
    "code": "BMSUserErrorInvalidInput",
    "message": "Input provided for the call is invalid. Please check the required inputs",
    "target": null,
    "details": null,
    "innerError": null
  }
}

Caught exception, type: System.AggregateException
Handling aggregate exception
Received CloudException, ErrorCode: BMSUserErrorInvalidInput, Message: Input provided for the call is invalid. Please 
check the required inputs
Restore-AzRecoveryServicesBackupItem : Input provided for the call is invalid. Please check the required inputs
At C:\scripts\RestoreDBFromVault.ps1:93 char:5
+     Restore-AzRecoveryServicesBackupItem -WLRecoveryConfig $AnotherIn ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Restore-AzRecoveryServicesBackupItem], CloudException
    + FullyQualifiedErrorId : BMSUserErrorInvalidInput,Microsoft.Azure.Commands.RecoveryServices.Backup.Cmdlets.Restor 
   eAzureRmRecoveryServicesBackupItem

AzureQoSEvent: Module: Az.RecoveryServices:4.4.0; CommandName: Restore-AzRecoveryServicesBackupItem; PSVersion: 
5.1.14409.1005; IsSuccess: False; Duration: 00:00:00.7282681; Exception: Input provided for the call is invalid. 
Please check the required inputs;
Finish sending metric.
6:30:39 PM - RestoreAzureRmRecoveryServicesBackupItem end processing.
pvrk commented 3 years ago

@mwsutherland : Thanks for the debug information. We will start an internal investigation with this. Also will tackle both issues separately and provide clarity on entire restore process for SQL using Powershell. Thanks.

pvrk commented 3 years ago

@mwsutherland : Meanwhile can you check with Get-AzRecoveryServicesBackupRecoveryLogChain command and see if $BackupItem.LatestRecoveryPoint is within the range of the valid log chain?

mwsutherland commented 3 years ago

Very interesting. At the moment, the RecoveryPointTime of the last recovery point is Friday, October 1, 2021 7:34:07 PM the backup Item LatestRecoveryPoint is also Friday, October 1, 2021 7:34:07 PM but the Log chain End time is Friday, October 1, 2021 7:32:11 PM

At this point, I'm sure you will not be surprised that trying to restore at the PIT of 7:34:07 fails while restoring to a PIT of 7:32:11 works.

Does this mean that the last log backup went until 7:32:11 and the following full backup happened at 7:34:07? And that the restore with point in time is ignoring the new full backup?

mwsutherland commented 3 years ago

The solution has been found to my RecoveryPoint restore issue (I was including an unnecessary parameter, causing things not to work), so now I am able to restore to a recovery point. It works in my test case at least at the moment. I'll update my evening job and see how they like it.

mwsutherland commented 3 years ago

Both jobs started tonight!

hiaga commented 3 years ago

@mwsutherland : Please keep us posted and let us know if you could restore to both 1) Recovery Point 2) Log Point-In-Time

mwsutherland commented 3 years ago

When I found the differing times times above (recovery points being after log chain end time), the point-in-time restore to the recovery times failed while the restore to the end of long chain worked. My recovery point restores are working now that the problem with creating a proper config is resolved.

Adding a mention about how to get the valid range of times to the documentation about the -PointInTime parameter of Get-AzRecoveryServicesBackupWorkloadRecoveryConfig could be very useful to future users. An error message indicating that the pointInTime value is out of range would be awesome.

pvrk commented 3 years ago

@mwsutherland : Apologies for the delay in response and thanks for the feedback. We have 2 actions items from this investigation: 1) Clearly mention in the documentation about how to get log point-in-time restores and the required combinations - This is taken up and will be reflected by Oct 19th. In the long run, we will try to see if we can throw an error (by using parameter sets or just throw an error) 2) Throw a relevant error message (that given point-in-time is beyond log chain) and this is added to the backlog. I don't have definite timelines yet. I will update this bug when I have them.

Closing this issue. Please re-open if these actions are not enough.