Unable to deploy DSC extension

[francescomagalini]

Hi, I'm trying to define a VM with a DSC extension that should register the VM with Azure Automation

Here is the fluent definition:

vm = azure.VirtualMachines
  .Define(vmName)
  .WithRegion(region)
  .WithExistingResourceGroup(rg)
  .WithExistingPrimaryNetworkInterface(nic)
  .WithLatestWindowsImage ("MicrosoftWindowsServer", "WindowsServer", "PlatformImage")
  .WithAdminUsername(adminUserName)
  .WithAdminPassword(adminPassword)

  .WithOSDiskStorageAccountType(StorageAccountTypes.PremiumLRS)
  .WithDataDiskDefaultStorageAccountType(StorageAccountTypes.StandardLRS)
  .WithDataDiskDefaultCachingType(CachingTypes.ReadOnly)
  .WithOSDiskCaching(CachingTypes.ReadWrite)
  .WithOSDiskName(osDiskName)
  .WithOSDiskSizeInGB(128)

  .WithExistingAvailabilitySet(avset)
  .WithSize(role.InstanceType)

  .DefineNewExtension(vmName)
    .WithPublisher("Microsoft.Powershell")
    .WithType("DSC")
    .WithVersion("2.75")
    .WithMinorVersionAutoUpgrade()

    .WithPublicSetting("configurationArguments", new Dictionary<object, object>
                                                {
                                                    {"RegistrationKey", new Dictionary<string,object>
                                                        {
                                                            { "UserName", "PLACEHOLDER_DONOTUSE" },
                                                            { "Password", "PrivateSettingsRef:RegistrationKeyPrivate"}
                                                        }
                                                    },
                                                    { "RegistrationUrl", "endpoint" },
                                                    { "NodeConfigurationName", "name" }
                                                })
    .WithProtectedSetting("Items", new Dictionary<string, string> {
                                { "RegistrationKeyPrivate", "key" }
                          })
    .Attach()

  .Create();

I get: [ERROR] Registration of the Dsc Agent with the server https://we-agentservice-prod-1.azure-automation.net/accounts/41fe028f-c44d-475c-8e51-ee93e0251c51 failed. The underlying error is: The attempt to register Dsc Agent with AgentId 58F3E916-47CD-11E8-B04D-000D3A2B8279 with the server https://we-agentservice-prod-1.azure-automation.net/accounts/41fe028f-c44d-475c-8e51-ee93e0251c51/Nodes(AgentId='58F3E916-47CD-11E8-B04D-000D3A2B8279') returned unexpected response code Unauthorized. full instance view:

{
  "platformUpdateDomain": 0,
  "platformFaultDomain": 0,
  "vmAgent": {
    "vmAgentVersion": "2.7.41491.875",
    "statuses": [
      {
        "code": "ProvisioningState/succeeded",
        "level": "Info",
        "displayStatus": "Ready",
        "message": "GuestAgent is running and accepting new configurations.",
        "time": "2018-04-24T14:45:45+00:00"
      }
    ],
    "extensionHandlers": [
      {
        "type": "Microsoft.Powershell.DSC",
        "typeHandlerVersion": "2.75.0.0",
        "status": {
          "code": "ProvisioningState/succeeded",
          "level": "Info",
          "displayStatus": "Ready"
        }
      }
    ]
  },
  "disks": [
    {
      "name": "ifx76adfs1_os",
      "statuses": [
        {
          "code": "ProvisioningState/succeeded",
          "level": "Info",
          "displayStatus": "Provisioning succeeded",
          "time": "2018-04-24T14:35:16.5671781+00:00"
        }
      ]
    }
  ],
  "extensions": [
    {
      "name": "ifx76adfs1",
      "type": "Microsoft.Powershell.DSC",
      "typeHandlerVersion": "2.75.0.0",
      "substatuses": [
        {
          "code": "ComponentStatus/DscConfigurationLog/failed/8",
          "level": "Error",
          "displayStatus": "Provisioning failed",
          "message": "[2018-04-24 14:40:04Z] [VERBOSE] Performing the operation \"Start-DscConfiguration: SendMetaConfigurationApply\" on target \"MSFT_DSCLocalConfigurationManager\".\r\n[2018-04-24 14:40:04Z] [VERBOSE] Perform operation 'Invoke CimMethod' with following parameters, ''methodName' = SendMetaConfigurationApply,'className' = MSFT_DSCLocalConfigurationManager,'namespaceName' = root/Microsoft/Windows/DesiredStateConfiguration'.\r\n[2018-04-24 14:40:04Z] [VERBOSE] An LCM method call arrived from computer ifx76adfs1 with user sid S-1-5-18.\r\n[2018-04-24 14:40:04Z] [VERBOSE] [ifx76adfs1]: LCM:  [ Start  Set      ]\r\n[2018-04-24 14:40:04Z] [VERBOSE] [ifx76adfs1]: LCM:  [ Start  Resource ]  [MSFT_DSCMetaConfiguration]\r\n[2018-04-24 14:40:04Z] [VERBOSE] [ifx76adfs1]: LCM:  [ Start  Set      ]  [MSFT_DSCMetaConfiguration]\r\n[2018-04-24 14:40:04Z] [VERBOSE] [ifx76adfs1]: LCM:  [ End    Set      ]  [MSFT_DSCMetaConfiguration]  in 0.0470 seconds.\r\n[2018-04-24 14:40:04Z] [VERBOSE] [ifx76adfs1]: LCM:  [ End    Resource ]  [MSFT_DSCMetaConfiguration]\r\n[2018-04-24 14:40:07Z] [VERBOSE] [ifx76adfs1]: LCM:  [ End    Set      ]\r\n[2018-04-24 14:40:08Z] [ERROR] Registration of the Dsc Agent with the server https://we-agentservice-prod-1.azure-automation.net/accounts/41fe028f-c44d-475c-8e51-ee93e0251c51 failed. The underlying error is: The attempt to register Dsc Agent with AgentId 58F3E916-47CD-11E8-B04D-000D3A2B8279 with the server https://we-agentservice-prod-1.azure-automation.net/accounts/41fe028f-c44d-475c-8e51-ee93e0251c51/Nodes(AgentId='58F3E916-47CD-11E8-B04D-000D3A2B8279') returned unexpected response code Unauthorized. .\r\n[2018-04-24 14:40:08Z] [VERBOSE] Operation 'Invoke CimMethod' complete.\r\n[2018-04-24 14:40:08Z] [VERBOSE] Set-DscLocalConfigurationManager finished in 3.963 seconds."
        },
        {
          "code": "ComponentStatus/DscExtensionLog/failed/8",
          "level": "Error",
          "displayStatus": "Provisioning failed",
          "message": "[2018-04-24 14:39:53Z] Updating execution status (HKLM:\\SOFTWARE\\Microsoft\\Azure\\DSC\\2.75.0.0\\Status)\r\n[2018-04-24 14:39:53Z] Transitioning to ProcessConfiguration state ...\r\n[2018-04-24 14:39:53Z] Getting handler execution status HKLM:\\SOFTWARE\\Microsoft\\Azure\\DSC\\2.75.0.0\\Status ...\r\n[2018-04-24 14:39:53Z] Getting handler execution status HKLM:\\SOFTWARE\\Microsoft\\Azure\\DSC\\2.75.0.0\\Status ...\r\n[2018-04-24 14:39:53Z] Settings handler status to 'transitioning' (C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\Status\\0.status)\r\n[2018-04-24 14:39:53Z] Applying DSC configuration:\r\n[2018-04-24 14:39:53Z]     Sequence Number:              0\r\n[2018-04-24 14:39:53Z]     Configuration URL:            C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\bin\\..\\scripts\\dscaa\\RegistrationMetaConfigV2.ps1\r\n[2018-04-24 14:39:53Z]     Configuration Script:         RegistrationMetaConfigV2.ps1\r\n[2018-04-24 14:39:53Z]     Configuration Function:       RegistrationMetaConfigV2\r\n[2018-04-24 14:39:53Z] Creating Working directory: C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\bin\\..\\DSCWork\\RegistrationMetaConfigV2.0\r\n[2018-04-24 14:39:53Z] Downloading configuration package\r\n[2018-04-24 14:39:53Z] Downloading C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\bin\\..\\scripts\\dscaa\\RegistrationMetaConfigV2.ps1 to C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\bin\\..\\DSCWork\\RegistrationMetaConfigV2.0\\RegistrationMetaConfigV2.ps1\r\n[2018-04-24 14:39:54Z] Looking for the definition of the configuration function.\r\n[2018-04-24 14:39:54Z] Executing C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\bin\\..\\DSCWork\\RegistrationMetaConfigV2.0\\RegistrationMetaConfigV2.ps1\r\n[2018-04-24 14:39:54Z] Preparing configuration arguments and configuration data.\r\n[2018-04-24 14:39:55Z] Created encryption certificate 8DFC56739E67935626ECB02B3D7E4A4D06F9C089\r\n[2018-04-24 14:39:55Z] Adding encryption certificate 8DFC56739E67935626ECB02B3D7E4A4D06F9C089 to the configuration data\r\n[2018-04-24 14:39:55Z] Creating MOF files.\r\n[2018-04-24 14:39:55Z] Executing the configuration function to generate the MOF files.\r\n[2018-04-24 14:39:56Z] Meta configuration found. Injecting Thumbprint.\r\n[2018-04-24 14:39:56Z] CertificateID not present in existing meta configuration; inserting 8DFC56739E67935626ECB02B3D7E4A4D06F9C089.\r\n[2018-04-24 14:39:56Z] Verifying metaconfiguration for reboot information...\r\n[2018-04-24 14:39:56Z] Executing Set-DscLocalConfigurationManager...\r\n[2018-04-24 14:39:59Z] Settings handler status to 'transitioning' (C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\Status\\0.status)\r\n[2018-04-24 14:40:10Z] Settings handler status to 'transitioning' (C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\Status\\0.status)\r\n[2018-04-24 14:40:10Z] Updating execution status (HKLM:\\SOFTWARE\\Microsoft\\Azure\\DSC\\2.75.0.0\\Status)\r\n[2018-04-24 14:40:10Z] DSC configuration completed.\r\n[2018-04-24 14:40:10Z] No meta mof back up file exist to restore ...\r\n[2018-04-24 14:40:10Z] Settings handler status to 'error' (C:\\Packages\\Plugins\\Microsoft.Powershell.DSC\\2.75.0.0\\Status\\0.status)"
        },
        {
          "code": "ComponentStatus/Metadata/succeeded",
          "level": "Info",
          "displayStatus": "Provisioning succeeded",
          "message": "AgentID=58F3E916-47CD-11E8-B04D-000D3A2B8279;VMUUID=A215C9CD-56BA-430C-96F9-B5D76ED46A47;AzureResourceId=subscriptions/d93e08d7-e23f-44d1-bcae-fae548b15f12/resourceGroups/Dynamics365OnPremiseifx76_2018-04-24_16-33-03/providers/Microsoft.Compute/virtualMachines/ifx76adfs1"
        }
      ],
      "statuses": [
        {
          "code": "ProvisioningState/failed/8",
          "level": "Error",
          "displayStatus": "Provisioning failed",
          "message": "DSC Configuration 'RegistrationMetaConfigV2' completed with error(s). Following are the first few: Registration of the Dsc Agent with the server https://we-agentservice-prod-1.azure-automation.net/accounts/41fe028f-c44d-475c-8e51-ee93e0251c51 failed. The underlying error is: The attempt to register Dsc Agent with AgentId 58F3E916-47CD-11E8-B04D-000D3A2B8279 with the server https://we-agentservice-prod-1.azure-automation.net/accounts/41fe028f-c44d-475c-8e51-ee93e0251c51/Nodes(AgentId='58F3E916-47CD-11E8-B04D-000D3A2B8279') returned unexpected response code Unauthorized. .",
          "time": "2018-04-24T14:40:10+00:00"
        }
      ]
    }
  ],
  "statuses": [
    {
      "code": "ProvisioningState/succeeded",
      "level": "Info",
      "displayStatus": "Provisioning succeeded",
      "time": "2018-04-24T14:40:46.1463262+00:00"
    },
    {
      "code": "PowerState/running",
      "level": "Info",
      "displayStatus": "VM running"
    }
  ]
}

Parameters are correct. Any guess?

[inghak]

We have the same type of problem. We have several nodes in a cluster where the nodes are deployed with the same ARM template and DSC configuration. One of the nodes is causing this problem, the others not. So it is not the configuration that is the problem. The node causing the problem worked previously.

If I uninstall the DSC Extension (2.76) it will not be able to reinstall and it fails with this error. I uninstall and reinstall using the portal. I think somthing must be cleared properly somewhere, but I cannot figure out what. It is not acceptable to have to delete the node. This problem comes back periodically. Some cache somewhere, or something that should be deleted from node to "clear" properly?

[DanielFrei64]

I too have the exact same scenario as you inghak, Several nodes, all previously worked, using Azure Automation. 10 days ago I noticed it stopped working, cannot get it to reconnect. Exact same error message. Let me know if you've figured anything out. I have a support ticket with Microsoft and so far their best suggestion is to restart the server.

[MarcelZehner]

Same issue here with some brand new Azure Windows Server 2016 VMs. I am however able to add the VMs to Azure Automation DSC configs through the portal - with the exact same configurations I use in the ARM template. Strange.

[Dhawal05]

Any updates on this one. I am facing same issue and I need to do it within ARM template unfortunately.

[JovialJerboa]

We're experiencing the same issue, has anyone had any luck fixing this problem consistently?

[DanFrei64]

For me the issue was a bad automation account certificate on the server. I went to the computers certificate store and deleted all of the automation account certs and I was able to deploy the extension then.

[JovialJerboa]

Thanks for the tip, I’ll try that and let you know how I get on.

[JovialJerboa]

@DanFrei64 Thanks for your suggestion, this didn't work for me but for anyone who does need to remove certificates (I believe by default they only last 12 months) I've written a powershell one liner for this. You will need to have administrator privileges: Get-ChildItem -Recurse -Path cert: | Where-Object { $_.Subject -like '*AzureDSCExtension*' } | ForEach-Object { $_ | Remove-Item -Force }

We did solve our problem which was actually unrelated to the DSC systems at all. We had created a private load balancer and added the machine to the backend pool, which prevents all outbound connections to the internet. Nodes connecting to an Azure DSC Pull Server require access to a publicly facing web service, in our case "https://uks-agentservice-prod-1.azure-automation.net".

We solved this problem by adding another public load balancer as described in the documentation here