search

stgeorgi / wvdquickstart

26 stars 31 forks source link

CreateDevopsPipeline failed due to in preview commands #7

Closed penicaudm closed 3 years ago

penicaudm commented 3 years ago

Running the quickstart template fails at the devops pipeline creation due to warnings from the preview API version

Here is the attached Status message 

{
    "status": "failed",
    "error": {
        "code": "ResourceDeploymentFailure",
        "message": "The resource operation completed with terminal provisioning state 'failed'.",
        "details": [
            {
                "code": "DeploymentScriptError",
                "message": "The provided script failed with multiple errors. First error:\r\nThis command is in preview. It may be changed/removed in a future release.. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more deployment script information.",
                "details": [
                    {
                        "code": "DeploymentScriptError",
                        "message": "This command is in preview. It may be changed/removed in a future release."
                    },
                    {
                        "code": "DeploymentScriptError",
                        "message": "WARNING: Successfully created a pipeline with Name: WVD QuickStart, Id: 1."
                    },
                    {
                        "code": "DeploymentScriptError",
                        "message": "ERROR: Could not queue the build because there were validation errors or warnings."
                    }
                ]
            }
        ]
    }
}

The provisioning state returns failed and status Conflict The azure devops pipeline has been successfully created however.

I am trying to see if the ARM template or API version can be changed to suppress the warnings or see if there's an underlying issue and will report here my findings.

Regards

pazdedav commented 3 years ago

I could be wrong @stgeorgi , but I believe that deployment scripts (ARM templates feature) will be in GA in October 2020 and it has been running in a preview (customers had to sign for and their subscriptions had to be whitelisted, otherwise they couldn't be used).

In other words, it might not be available for all customers yet, so this could be the reason this step fails for @penicaudm .

stgeorgi commented 3 years ago

@pazdedav you are correct that deployment scripts is in preview. However we haven't had the need to white list any of the subs we used during deployment. Further in the month of August we have 45+ deployments (not internal) with success rate of 97%. I also checked the documentation pages and talks nothing about white listing https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-script-template?tabs=CLI

penicaudm commented 3 years ago

@pazdedav @stgeorgi thank you for these elements. I believe the warnings come from the fact that some of these commands are in preview or have been changed, which sends warning (just like the Poweshell Modules from Microsoft, on which you can suppress warnings through a global variable AFAIK).

I believe the issue here is that the deployment fails due to the warnings, even though none of them have an impact on the ressources, and am not sure of how I can resolve them.

Cheers

stgeorgi commented 3 years ago

Can you share what warning you get?

penicaudm commented 3 years ago

Here are all the details I can think of: This is a dev/test tenant we use, I was doing a POC on WVD with quickstart I am using the admin account on the tenant without MFA A domain-joiner account has the correct permissions (GA and DC admin on AAD DS) and no MFA either.

Region is west eu in a RG made for this deployment.

[
  {
    "id": "/subscriptions/REDACTED/resourceGroups/wvd-westeu-rg/providers/Microsoft.Resources/deployments/Microsoft.Template-20200924102332/operations/662AFE511B507F61",
    "operationId": "662AFE511B507F61",
    "properties": {
      "provisioningOperation": "Create",
      "provisioningState": "Failed",
      "timestamp": "2020-09-24T08:30:19.6501021Z",
      "duration": "PT4M10.0525225S",
      "trackingId": "6fc68b0e-275d-459e-9133-689a633f43f3",
      "statusCode": "Conflict",
      "statusMessage": {
        "status": "failed",
        "error": {
          "code": "ResourceDeploymentFailure",
          "message": "The resource operation completed with terminal provisioning state 'failed'.",
          "details": [
            {
              "code": "DeploymentScriptError",
              "message": "The provided script failed with multiple errors. First error:\r\nThis command is in preview. It may be changed/removed in a future release.. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more deployment script information.",
              "details": [
                {
                  "code": "DeploymentScriptError",
                  "message": "This command is in preview. It may be changed/removed in a future release."
                },
                {
                  "code": "DeploymentScriptError",
                  "message": "WARNING: Successfully created a pipeline with Name: WVD QuickStart, Id: 1."
                },
                {
                  "code": "DeploymentScriptError",
                  "message": "ERROR: Could not queue the build because there were validation errors or warnings."
                }
              ]
            }
          ]
        }
      },
      "targetResource": {
        "id": "/subscriptions/REDACTED/resourceGroups/wvd-westeu-rg/providers/Microsoft.Resources/deploymentScripts/createDevopsPipeline",
        "resourceType": "Microsoft.Resources/deploymentScripts",
        "resourceName": "createDevopsPipeline"
      }
    }
  },
{
    "status": "failed",
    "error": {
        "code": "ResourceDeploymentFailure",
        "message": "The resource operation completed with terminal provisioning state 'failed'.",
        "details": [
            {
                "code": "DeploymentScriptError",
                "message": "The provided script failed with multiple errors. First error:\r\nThis command is in preview. It may be changed/removed in a future release.. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more deployment script information.",
                "details": [
                    {
                        "code": "DeploymentScriptError",
                        "message": "This command is in preview. It may be changed/removed in a future release."
                    },
                    {
                        "code": "DeploymentScriptError",
                        "message": "WARNING: Successfully created a pipeline with Name: WVD QuickStart, Id: 1."
                    },
                    {
                        "code": "DeploymentScriptError",
                        "message": "ERROR: Could not queue the build because there were validation errors or warnings."
                    }
                ]
            }
        ]
    }
}

All other operations are okay:

image

pazdedav commented 3 years ago

@pazdedav you are correct that deployment scripts is in preview. However we haven't had the need to white list any of the subs we used during deployment. Further in the month of August we have 45+ deployments (not internal) with success rate of 97%. I also checked the documentation pages and talks nothing about white listing https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-script-template?tabs=CLI

I must have mixed that up with templateSpecs private preview, which requires whitelisting. Sorry for the confusion @stgeorgi

stgeorgi commented 3 years ago

@penicaudm can you please look at the Azure Automation account and share if any of the jobs there have failed?

penicaudm commented 3 years ago

@stgeorgi Yes

image

3 errors on devOpsSetupRunbook (first one is runbook already exists, because it was my third deployment. previous ones I used incorrect values) The last 2 errors for this one are bad requests (400)

Invoke-RestMethod : {"$id":"1","innerException":null,"message":"TF400948: A Git repository with the name WVDQuickStartProj082336Z already exists.",
"typeName":"Microsoft.TeamFoundation.Git.Server.GitRepositoryNameAlreadyExistsException, 
Microsoft.TeamFoundation.Git.Server","typeKey":"GitRepositoryNameAlreadyExistsException","errorCode":0,"eventId":3000} At line:186 char:13 + $response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "B ... + 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
Invoke-RestMethod : {"$id":"1","innerException":null,"message":"TF400898: An Internal Error Occurred. Activity Id: 7774ab28-6668-4dde-892d-8e94be3d8da3.","typeName":"Newtonsoft.Json.JsonReaderException, Newtonsoft.Json","typeKey":"JsonReaderException","errorCode":0,"eventId":0} At line:433 char:13 + $response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "B ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
Invoke-RestMethod : {"$id":"1","innerException":null,"message":"Permissions request cannot be null","typeName":"System.InvalidOperationException, mscorlib","typeKey":"InvalidOperationException","errorCode":0,"eventId":0} At line:457 char:13 + $response = Invoke-RestMethod -Method PATCH -Uri $url -Headers @{Auth ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

ServicePrincipalRunbook has no errors

inputValidationRunbook has an error due to a missing automation variable and 17 errors that say

"Cannot connect to CIM server. The specified service does not exist as an installed service."

I'll try to provide more details tomorrow, and also destroy all the resources and try again.

Cheers

stgeorgi commented 3 years ago

so based on the error TF400898 it seems that there were issues with permissions to dev ops. if you are redeploying the entire thing (not just re-running the pipeline) please make sure you do this

Go to Azure Active Directory –> App registrations and remove the “WVDServicePrincipal” Go to Azure Active Directory –> Groups and remove the “WVDTestUsers” group Go to Azure Active Directory –> Users and remove the “WVDTestUser001” user profile Navigate to Azure DevOps and delete the WVD QuickStart organization by going to the organization’s settings and scrolling to the bottom of the page.

penicaudm commented 3 years ago

Hi, sorry I didn't get back to you sooner. I tried on a brand new azure tenant and still had error, instead went with Terraform some manual stuff because I ran out of time to finish the POC. Unfortunately, I don't have more details, but I'll give it another try whenever I can and open another issue if necessary.

Cheers