search

pulumi / actions

Deploy continuously to your cloud of choice, using your favorite language, Pulumi, and GitHub!
Apache License 2.0
240 stars 73 forks source link

`pulumi preview` is ignoring `secret-provider` parameter #1170

Open pcyang opened 1 month ago

pcyang commented 1 month ago

What happened?

I followed the instruction in the Readme and passed in my gcpkms secret-provider, but I'm still getting error that ask me for PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE https://github.com/pulumi/actions?tab=readme-ov-file#configuration

My example github action step is in the example section below.

Here's the full debug error:

##[debug]Evaluating condition for step: 'Preview infrastructure changes 📋'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Preview infrastructure changes 📋
##[debug]Loading inputs
##[debug]Evaluating: github.token
##[debug]Evaluating Index:
##[debug]..Evaluating github:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'token'
##[debug]=> '***'
##[debug]Result: '***'
##[debug]Loading env
Run pulumi/actions@v4
##[debug]Configuration is loaded
##[debug]Platform: linux-x64
Configured range: ^3
/usr/local/bin/pulumi version
v3.116.0
warning: A new version of Pulumi is available. To upgrade from version '3.116.0' to '3.116.1', visit https://pulumi.com/docs/install/ for manual instructions and release notes.
Pulumi version 3.116.0 is already installed on this machine. Skipping download
Logging into gs://MY_GCP_PULUMI_STATE_BUCKET
##[debug]Working directory resolved at /home/runner/work/MY_REPO/MY_REPO
::group::pulumi preview on MY_STACK_NAME
pulumi preview on MY_STACK_NAME
  ##[debug]Running action preview
  /home/runner/work/_actions/pulumi/actions/v4/webpack:/pulumi-github-action/node_modules/@pulumi/pulumi/automation/errors.js:77
                      : new CommandError(result);
  ^
  CommandError: code: -2
   stdout: 
   stderr: Command failed with exit code 255: pulumi preview --exec-agent pulumi/actions@v3 --color auto --exec-kind auto.local --event-log /tmp/automation-logs-preview-IgkP4w/eventlog.txt --stack MY_STACK_NAME --non-interactive
  error: getting stack configuration: get stack secrets manager: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables
   err?: Error: Command failed with exit code 255: pulumi preview --exec-agent pulumi/actions@v3 --color auto --exec-kind auto.local --event-log /tmp/automation-logs-preview-IgkP4w/eventlog.txt --stack MY_STACK_NAME --non-interactive
  error: getting stack configuration: get stack secrets manager: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables
      at Object.createCommandError (/home/runner/work/_actions/pulumi/actions/v4/webpack:/pulumi-github-action/node_modules/@pulumi/pulumi/automation/errors.js:77:1)
      at Object.<anonymous> (/home/runner/work/_actions/pulumi/actions/v4/webpack:/pulumi-github-action/node_modules/@pulumi/pulumi/automation/cmd.js:76:1)
      at Generator.throw (<anonymous>)
      at rejected (/home/runner/work/_actions/pulumi/actions/v4/webpack:/pulumi-github-action/node_modules/@pulumi/pulumi/automation/cmd.js:19:1)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)
  ##[debug]Node Action run completed with exit code 1
  ##[debug]Finishing: Preview infrastructure changes 📋

Am I passing it incorrectly? how do I get pulumi preview command to use my custom secret-provider?

Example

      - name: Preview infrastructure changes 📋
        uses: pulumi/actions@v4
        with:
          command: preview
          stack-name: MY_STACK_NAME
          cloud-url: gs://MY_GCP_PULUMI_STATE_BUCKET
          secrets-provider: gcpkms://projects/MY_PROJECT/locations/MY_REGION/keyRings/MY_KEY_RINGS/cryptoKeys/MY_KEY

Output of pulumi about

Cannot do pulumi about in github action.

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

pcyang commented 1 month ago

Adding more context -- Pulumi.MY_STACK_NAME.yaml is not checked in to my repository. Locally, I need to do pulumi config refresh before I run the pulumi preview command. It seems like I'd need way to be able to do that in pulumi action.

pcyang commented 1 month ago

Okay, looks like I can just do a pulumi config refresh right before it as a workaround for now

      - name: Pulling remote config for Pulumi 🛠️
        run: |
          pulumi stack select MY_STACK_NAME
          pulumi config refresh

      - name: Preview infrastructure changes summary 📋
        uses: pulumi/actions@v4
        with:
          command: preview
          stack-name: MY_STACK_NAME
          cloud-url: gs://MY_GCP_PULUMI_STATE_BUCKET
justinvp commented 1 month ago

@pcyang, out of curiosity, why aren't you checking in Pulumi.MY_STACK_NAME.yaml?

The secrets-provider option primarily exists for use with the upsert option (see https://github.com/pulumi/actions/issues/338):

upsert - (optional) Allows the creation of the specified stack if it currently doesn't exist. PLEASE NOTE: This will create a Pulumi.<stack-name>.yaml file that you will need to add back to source control as part of the action if you wish to perform any further tasks with that stack.

pcyang commented 1 month ago

@justinvp Thank you for the response!

We have decided against checking in Pulumi.MY_STACK_NAME.yaml for the following reasons

  1. We don't feel comfortable checking in the encryptedKey included in Pulumi.MY_STACK_NAME.yaml for secret that were encrypted using our GCP KMS Key. Encrypting the credential doesn't lower the data classification/sensitivity of the underlying content, we don't want to have that in our repository, public or private. a. We disagree with the approach and prefer using Secret Manager solution instead. We were under the assumption that this key is only used to encrypt secret configuration specifically, and not for encrypting the rest of the stack config in the backend bucket. b. We don't have a mechanism to disable this feature, and just using passphrase requires us to manually type in a fake/dummy/blank password or commit it to ENV even if we want to ignore the feature. c. It also appears that key rotation triggered from GCP KMS isn't recognized by Pulumi, so we'd have to manually rotate by calling pulumi stack change-secrets-provider.

  2. We wanted to have hierarchical configuration, but that is currently not supported in Pulumi with Self-Managed backend state, only with Pulumi ESC. Pulumi configuration out of the box is pretty confusing with Pulumi.yaml only copied during new stack creation, and not applying to existing stack on changes. In our case, we basically find the base Pulumi.yaml and Pulumi.MY_STACK_NAME.yaml useless other than just stack name creation, and we simply load adobe/himl on startup and use that instead, and track the change versioning in separate config/ files.

  3. As state in Configuration and Secrets documentation, checking in these file is optional, and we can always fetch them using pulumi config refresh, we see no reason for us to check in that file.

Could the documentation be updated to note that the secrets-provider parameter is only use for upsert and will be ignored for other case? Also would be nice if it mentioned the need to call pulumi config refresh if the user didn't check in their stack yaml file.