PULUMI_ACCESS_TOKEN required when using Azure storage backend

[aonyxrocks]

Describe the bug PULUMI_ACCESS_TOKEN must be set despite an Azure storage account being configured as the backend in Pulumi.yaml.

To Reproduce

1. Have a service connection between Azure DevOps and an Azure tenant + subscription.
2. Create an Azure storage account in that subscription, key authentication disabled (effectively allowing only RBAC).
3. Create a blob container for pulumi.
4. Assign Storage Blob Data Contributor permissions to the service connection principal at the container scope.
5. Create a new pulumi project in a new repo.
6. Edit Pulumi.yaml to set the backend url to point to the storage blob container:

   backend:
    url: azblob://<container>?storage_account=<storage_account>

7. Create a pipeline to run up command using the above service connection --> task exits with error message:

   [error]Couldn't determine which login method to use. This task extension supports Pulumi Service backend and self-managed backends. Learn more at https://www.pulumi.com/docs/intro/concepts/state.

8. Set environment variable PULUMI_ACCESS_TOKEN to a valid token for the pipeline.
9. Run pipeline again --> task logs in using Pulumi access token, but proceeds to use the storage backend and deploys resources successfully.

Expected behavior Pulumi task determines login method from the Pulumi.yaml and uses storage account + service connection to authenticate, without the need to set PULUMI_ACCESS_TOKEN or any other environment variable (e.g. AZURE_STORAGE_KEY, AZURE_STORAGE_SAS_TOKEN, etc.).

Additional context I am aware this only works when the backend and deployed resources are in the same tenant (and subscription). I imagine a dedicated property to use a different service connection to access the backend or a dedicated "pulumi login" pipeline task could help with different scenarios, but that is beside the point.

An Azure key vault as the secrets provider is part of the same setup but was not tested independently.

[RobbieMcKinstry]

Interesting! I suppose the pipeline task could attempt pulumi login, and skip checking for the PULUMI_ACCESS_TOKEN if login is successful. Thanks for the thorough write-up!

[jlewicki-nevo]

I have encountered a related issue.

Similarly to @aonyxrocks I have a stack configured to use Azure storage backend. I have a Azure pipeline template that invokes the pulumi task like so:

- task: Pulumi@1
  inputs:
    azureSubscription: ${{ parameters.subscriptionName }}
    command: ${{ parameters.pulumiCommand }}
    args: ${{ parameters.pulumiArgs }}
    stack:  ${{ parameters.environmentName }}
  env:
    AZURE_STORAGE_CONTAINER: $(storageContainer)
    AZURE_STORAGE_ACCOUNT: $(storageAccount)
    AZURE_STORAGE_KEY: $(storageAccessKey)

Note that I am setting AZURE_STORAGE_ environment variables (env), not pipeline variables in a variables map.

With this configuration, I am getting: Couldn't determine which login method to use. This task extension supports Pulumi Service backend and self-managed backends

This appears to be caused by this line:

const azureStorageContainer = agentEnvVars["AZURE_STORAGE_CONTAINER"];

It appears to me that the code is checking for AZURE_STORAGE_CONTAINER in a map that is populated from Task pipeline variables, not the process environment variables as I was expecting (and how the Pulumi CLI works). Maybe instead of agentEnvVars, this line could check loginEnvVars instead (which does appear to include the process env).

Given my pipeline organization it is definitely more convenient to be able to use the env map for the Pulumi task

[rhuanbarreto]

I'm facing the same issue but with a different perspective. My Pulumi program uses the PULUMI_ACCESS_TOKEN to fetch data from Pulumi cloud in order to get some info using the REST API. So the task doesn't expose the pulumi.access.token variable to the environment as PULUMI_ACCESS_TOKEN. Then the pulumi program fails due to this lack.

I think this is the improvement that must be done.

[reubano]

Can confirm similar issue as both OP and @jlewicki-nevo. The AZURE_STORAGE_CONTAINER issue may also be related to #48. This is what seems to work for me.

variables:
  - name: AZURE_STORAGE_CONTAINER
    value: <YOUR_STORAGE_CONTAINER>

...

- task: Pulumi@1
  inputs:
    azureSubscription: $(subscriptionConnectionName)
    command: up
    args: '--yes --logtostderr'
    cwd: $(cwd)
    stack: dev
  env:
    PULUMI_ACCESS_TOKEN: $(pulumiAccessToken)
    AZURE_STORAGE_ACCOUNT: $(storageAccount)