Azure / azure-dev

A developer CLI that reduces the time it takes for you to get started on Azure. The Azure Developer CLI (azd) provides a set of developer-friendly commands that map to key stages in your workflow - code, build, deploy, monitor, repeat.
https://aka.ms/azd
MIT License
393 stars 187 forks source link

[Issue] Error "panic: don't know how to prompt for type *survey.Password" on deploying Aspire solution from Azure Devops Pipeline #4151

Closed MateuszPeczek closed 1 month ago

MateuszPeczek commented 1 month ago

Output from azd version azd version 1.9.5 (commit cd2b7af9995d358aab33c782614f801ac1997dde)

Describe the bug After running azd pipeline config --provider azdo (based on steps in: https://learn.microsoft.com/en-us/dotnet/aspire/deployment/azure/aca-deployment-github-actions?tabs=windows&pivots=azure-pipelines) and deploying pipeline to DevOps project I'm receiving error on provisioning step:

Provisioning Azure resources (azd provision)
Provisioning Azure resources can take some time.

Analyzing Aspire Application (this might take a moment...)
Downloading Bicep
  (✓) Done: Downloading Bicep
Initialize bicep provider
Retrieving subscriptions...
panic: don't know how to prompt for type *survey.Password

goroutine 1 [running]:
github.com/azure/azure-dev/cli/azd/pkg/input.askOneNoPrompt({0x55f0e2abf660?, 0xc00021d4a0?}, {0x55f0e27ad880?, 0xc000479fd0?})
    /mnt/vss/_work/1/s/cli/azd/pkg/input/asker.go:72 +0x3e5
github.com/azure/azure-dev/cli/azd/pkg/input.(*AskerConsole).Prompt.func1(0xc00022a1a0)
    /mnt/vss/_work/1/s/cli/azd/pkg/input/console.go:607 +0x77
github.com/azure/azure-dev/cli/azd/pkg/input.(*AskerConsole).doInteraction(0xc00022a1a0, 0xc000641540)
    /mnt/vss/_work/1/s/cli/azd/pkg/input/console.go:1125 +0x1b0
github.com/azure/azure-dev/cli/azd/pkg/input.(*AskerConsole).Prompt(0xc00022a1a0, {0x55f0e2ac3168, 0xc00021bef0}, {{0xc0004b5040, 0x43}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...})
    /mnt/vss/_work/1/s/cli/azd/pkg/input/console.go:606 +0x395
github.com/azure/azure-dev/cli/azd/pkg/infra/provisioning/bicep.promptWithValidation[...]({0x55f0e2ac3168?, 0xc00021bef0}, {0x55f0e2ad34b0, 0xc00022a1a0}, {{0xc0004b5040, 0x43}, {0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
    /mnt/vss/_work/1/s/cli/azd/pkg/infra/provisioning/bicep/prompt.go:232 +0x119
github.com/azure/azure-dev/cli/azd/pkg/infra/provisioning/bicep.(*BicepProvider).promptForParameter(0xc0004a6fc0, {0x55f0e2ac3168, 0xc00021bef0}, {0xc000015c26, 0x9}, {{0xc000015c50, 0xc}, {0x0, 0x0}, 0x0, ...})
    /mnt/vss/_work/1/s/cli/azd/pkg/infra/provisioning/bicep/prompt.go:185 +0x100d
github.com/azure/azure-dev/cli/azd/pkg/infra/provisioning/bicep.(*BicepProvider).ensureParameters(0xc0004a6fc0, {0x55f0e2ac3168, 0xc00021bef0}, {{0xc000047080, 0x5b}, {0xc000015ba7, 0x7}, 0xc0005837d0, 0xc000583890, 0x0})
    /mnt/vss/_work/1/s/cli/azd/pkg/infra/provisioning/bicep/bicep_provider.go:2067 +0xc1b
github.com/azure/azure-dev/cli/azd/pkg/infra/provisioning/bicep.(*BicepProvider).EnsureEnv(0xc0004a6fc0, {0x55f0e2ac3168, 0xc00021bef0})
    /mnt/vss/_work/1/s/cli/azd/pkg/infra/provisioning/bicep/bicep_provider.go:164 +0x33a
github.com/azure/azure-dev/cli/azd/pkg/infra/provisioning/bicep.(*BicepProvider).Initialize(0xc0004a6fc0, {0x55f0e2ac3168, 0xc00021bef0}, {0xc000228300?, 0x0?}, {{0x55f0e21e59e2, 0x5}, {0xc0002122d0, 0x18}, {0x55f0e21e4f02, ...}, ...})
    /mnt/vss/_work/1/s/cli/azd/pkg/infra/provisioning/bicep/bicep_provider.go:121 +0x199
github.com/azure/azure-dev/cli/azd/pkg/infra/provisioning.(*Manager).Initialize(0xc0002402d0, {0x55f0e2ac3168, 0xc00021bef0}, {0xc000228300, 0x31}, {{0x55f0e21e59e2, 0x5}, {0xc0002122d0, 0x18}, {0x55f0e21e4f02, ...}, ...})
    /mnt/vss/_work/1/s/cli/azd/pkg/infra/provisioning/manager.go:69 +0x22d
github.com/azure/azure-dev/cli/azd/internal/cmd.(*ProvisionAction).Run(0xc00021d2c0, {0x55f0e2ac3168, 0xc00021bef0})
    /mnt/vss/_work/1/s/cli/azd/internal/cmd/provision.go:190 +0x37f
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*MiddlewareRunner).RunAction.func1({0x55f0e2ac3168, 0xc00021bef0})
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/middleware.go:126 +0x429
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*HooksMiddleware).Run(0xc0000a5720, {0x55f0e2ac3168, 0xc00021bef0}, 0xc0003c8e80)
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/hooks.go:53 +0x262
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*MiddlewareRunner).RunAction.func1({0x55f0e2ac3168, 0xc00021bef0})
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/middleware.go:109 +0x2b8
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*TelemetryMiddleware).Run(0xc000224580, {0x55f0e2ac3168?, 0xc00021a9f0?}, 0xc0003c8e80)
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/telemetry.go:78 +0x918
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*MiddlewareRunner).RunAction.func1({0x55f0e2ac3168, 0xc00021a9f0})
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/middleware.go:109 +0x2b8
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*ExperimentationMiddleware).Run(0x55f0e21f3e2b?, {0x55f0e2ac3168, 0xc00021a9f0}, 0xc0003c8e80)
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/experimentation.go:64 +0x4d6
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*MiddlewareRunner).RunAction.func1({0x55f0e2ac3168, 0xc00021a9f0})
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/middleware.go:109 +0x2b8
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*UxMiddleware).Run(0xc000013b18, {0x55f0e2ac3168, 0xc00021a9f0}, 0xc0003c8e80)
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/ux.go:32 +0xa9
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*MiddlewareRunner).RunAction.func1({0x55f0e2ac3168, 0xc00021a9f0})
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/middleware.go:109 +0x2b8
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*DebugMiddleware).Run(0xc000013ae8, {0x55f0e2ac3168, 0xc00021a9f0}, 0xc0003c8e80)
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/debug.go:48 +0x35e
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*MiddlewareRunner).RunAction.func1({0x55f0e2ac3168, 0xc00021a9f0})
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/middleware.go:109 +0x2b8
github.com/azure/azure-dev/cli/azd/cmd/middleware.(*MiddlewareRunner).RunAction(0xc00044fc00, {0x55f0e2ac3168, 0xc00021a9f0}, 0xc000017aa0, {0xc0002123f0, 0x14})
    /mnt/vss/_work/1/s/cli/azd/cmd/middleware/middleware.go:130 +0x155
github.com/azure/azure-dev/cli/azd/cmd.(*CobraBuilder).configureActionResolver.func1(0xc0004d0780, {0xc000224060?, 0x0, 0x1})
    /mnt/vss/_work/1/s/cli/azd/cmd/cobra_builder.go:132 +0x3a7
github.com/spf13/cobra.(*Command).execute(0xc0004d0780, {0xc000224050, 0x1, 0x1})
    /home/cloudtest/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:856 +0x694
github.com/spf13/cobra.(*Command).ExecuteC(0xc00048ec80)
    /home/cloudtest/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:974 +0x38d
github.com/spf13/cobra.(*Command).Execute(...)
    /home/cloudtest/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:902
github.com/spf13/cobra.(*Command).ExecuteContext(...)
    /home/cloudtest/go/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:895
main.main()
    /mnt/vss/_work/1/s/cli/azd/main.go:65 +0x24b
##[error]Script failed with exit code: 2
/usr/bin/az account clear

To Reproduce

  1. Run azd init --from-code --environment dev on AppHost project
  2. Run azd pipeline config --provider azdo and provide secrets for dependencies.
  3. Run pipeline

Expected behavior Provisioning of all services as in given example.

Environment Azure Devops Pipelines

Agent name: 'Hosted Agent'
Agent machine name: '...'
Current agent version: '3.241.0'
Operating System
Runner Image
Runner Image Provisioner
Current image version: '20240721.1.0'
Agent running as: 'vsts'
Prepare build directory.
Set build variables.
Download all required tasks.
Downloading task: Bash (3.241.1)
Downloading task: PowerShell (2.242.1)
Downloading task: AzureCLI (2.242.0)

VS 2022 v17.10.5

Additional context Devops Pipeline:

trigger:
  - main
  - master

pool:
  vmImage: ubuntu-latest

steps:

  - task: Bash@3
    displayName: Install azd
    inputs:
      targetType: 'inline'
      script: |
        curl -fsSL https://aka.ms/install-azd.sh | bash

  # azd delegate auth to az to use service connection with AzureCLI@2
  - pwsh: |
      azd config set auth.useAzCliAuth "true"
    displayName: Configure `azd` to Use AZ CLI Authentication.

  - task: Bash@3
    displayName: Install .NET Aspire workload
    inputs:
      targetType: 'inline'
      script: |
        dotnet workload install aspire

  - task: AzureCLI@2
    displayName: Provision Infrastructure
    inputs:
      azureSubscription: azconnection
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        cd src/IntegrationEngine.AppHost
        azd provision --no-prompt
    env:
      AZURE_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
      AZURE_ENV_NAME: $(AZURE_ENV_NAME)
      AZURE_LOCATION: $(AZURE_LOCATION)

  - task: AzureCLI@2
    displayName: Deploy Application
    inputs:
      azureSubscription: azconnection
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        cd src/IntegrationEngine.AppHost
        azd deploy --no-prompt
    env:
      AZURE_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
      AZURE_ENV_NAME: $(AZURE_ENV_NAME)
      AZURE_LOCATION: $(AZURE_LOCATION)

config.json from .env file (guids replaced by ...)

{
  "infra": {
    "parameters": {
      "appConfig": "vault://.../...",
      "keyVault": "vault://.../...",
      "landingZoneStorageAccount": "vault://.../...",
      "serviceBusApiSend": "vault://.../...",
      "serviceBusRuntimeListen": "vault://.../..."
    }
  },
  "vault": "..."
}

When providing values for secrets, I've tried both wrapping them in "" and without it without any change.

rajeshkamal5050 commented 1 month ago

@vhvb1989 @weikanglim can you take a look? related issue https://github.com/Azure/azure-dev/issues/3795

vhvb1989 commented 1 month ago

@MateuszPeczek , please add the env var:

AZD_INITIAL_ENVIRONMENT_CONFIG: $(AZD_INITIAL_ENVIRONMENT_CONFIG)

as mentioned here: https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/configure-devops-pipeline?tabs=azdo#create-a-pipeline-definition

When you run azd pipeline config, azd creates a secret with the content from the config.json in the environment. You need to use this env var for the first step that runs azd in the pipeline to make azd to re-construct the initial configuration based on it.

For example, for your pipeline, it would be:

- task: AzureCLI@2
    displayName: Provision Infrastructure
    inputs:
      azureSubscription: azconnection
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        cd src/IntegrationEngine.AppHost
        azd provision --no-prompt
    env:
      AZURE_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
      AZURE_ENV_NAME: $(AZURE_ENV_NAME)
      AZURE_LOCATION: $(AZURE_LOCATION)
      # This env var is required only by the first step that uses azd (provision, up, deploy, etc) to create the env config
      AZD_INITIAL_ENVIRONMENT_CONFIG: $(AZD_INITIAL_ENVIRONMENT_CONFIG)

Also, you can use the setup-azd@0 step to install azd, like here: https://github.com/Azure-Samples/todo-csharp-sql/blob/main/.azdo/pipelines/azure-dev.yml#L16C11-L16C22 (if you can add it to your Azdo org).

MateuszPeczek commented 1 month ago

Thanks @vhvb1989 , this solved my issue, It could be a good idea to give the same information as an example code in those two places. As you can see, the documentation page I've pasted originally misses that information about AZD_INITIAL_ENVIRONMENT_CONFIG variable.

flutteryscrubbers commented 1 month ago

Is there a way to retrieve the secret generated from the config.json other than running a azd pipeline config? That command creates quite a few resources that I have already stood up. I'm getting the same error that MateuszPeczek was seeing, but I'd rather not run the azd pipeline config command. I believe once I can get ahold of this secret, then adding it to my Azure DevOps library set with a key name of AZD_INITIAL_ENVIRONMENT_CONFIG will fix the error above in the same way it fixed the issue for MateuszPeczek.

vhvb1989 commented 1 month ago

@flutteryscrubbers , sure, you can manually set the secret. Try doing this:

*** If you don't see that section, you can manually add it. Use the name of the input parameters you defined in your AppHost

*** This key is to save secrets outside the repository (in local user's directory). If the vault key is present, it means that some of the parameter's value are secrets, and its value would look like:

image

image

You should convert it to something like:

image

lmk if you face any issues.

vhvb1989 commented 1 month ago

I created a new issue to allow folks not to use the AZD_INITIAL_ENVIRONMENT_CONFIG and define unique secrets/variables in CI for each input: https://github.com/Azure/azure-dev/issues/4162

microsoft-github-policy-service[bot] commented 1 month ago

Hi @MateuszPeczek, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

tonkatsu7 commented 4 days ago

@flutteryscrubbers , sure, you can manually set the secret. Try doing this:

  • Find out the environment name you are currently using. You can run azd env list. The default=true is the environment used by default when you don't set the e flag.
  • Find the config.json file locally at .azure/<environment name>/config.json and copy its content to a new file.
  • There should be an infra.parameters key, like: image

*** If you don't see that section, you can manually add it. Use the name of the input parameters you defined in your AppHost

  • Remove the key vault (if it is present), like image

*** This key is to save secrets outside the repository (in local user's directory). If the vault key is present, it means that some of the parameter's value are secrets, and its value would look like:

image

  • Replace any vault://... value with the actual value you want to set on CI/CD. Look at this quick example. For a config like:

image

You should convert it to something like:

image

  • Having your file ready, copy the content and create the secret for your CI provider (azdo or gh). The content of the secret is the raw json content.

lmk if you face any issues.

Hi @vhvb1989, sorry I don't follow.... how do I use a value or seret value form a YML variable or variable group? Thanks!