pulumi / pulumi

Pulumi - Infrastructure as Code in any programming language 🚀
https://www.pulumi.com
Apache License 2.0
21.52k stars 1.1k forks source link

Use of "requireSecret" in Typescript project results in module not found error #11257

Open 0xJem opened 1 year ago

0xJem commented 1 year ago

What happened?

I have the following setup:

You can browse the source code for the particular commit here: https://github.com/OlympusDAO/rbs-discord-alerts/tree/66b18abd5d91c7fdc59ce1f228ff84ed9c2db25b

The problem is that when the deployed function runs, it gives this error: Error: Cannot find module './output'

Steps to reproduce

Deploy the linked source code, run the function in GCF (not locally)

Expected Behavior

It should run. Running it locally is fine.

Actual Behavior

It complains about a module not found

Output of pulumi about

CLI Version 3.46.0 Go Version go1.19.2 Go Compiler gc

Plugins NAME VERSION gcp 6.41.0 nodejs unknown

Host OS darwin Version 13.0 Arch arm64

This project is written in nodejs: executable='/Users/XYZ/.nvm/versions/node/v16.14.0/bin/node' version='v16.14.0'

Current Stack: dev

TYPE URN pulumi:pulumi:Stack urn:pulumi:dev::rbs-discord-alerts::pulumi:pulumi:Stack::rbs-discord-alerts-dev gcp:cloudfunctions:CallbackFunction urn:pulumi:dev::rbs-discord-alerts::gcp:cloudfunctions:CallbackFunction::rbs-discord-alerts-dev pulumi:providers:gcp urn:pulumi:dev::rbs-discord-alerts::pulumi:providers:gcp::default_6_41_0 gcp:monitoring/notificationChannel:NotificationChannel urn:pulumi:dev::rbs-discord-alerts::gcp:monitoring/notificationChannel:NotificationChannel::discord gcp:monitoring/notificationChannel:NotificationChannel urn:pulumi:dev::rbs-discord-alerts::gcp:monitoring/notificationChannel:NotificationChannel::email gcp:firestore/document:Document urn:pulumi:dev::rbs-discord-alerts::gcp:firestore/document:Document::rbs-discord-alerts-dev gcp:storage/bucket:Bucket urn:pulumi:dev::rbs-discord-alerts::gcp:cloudfunctions:CallbackFunction$gcp:storage/bucket:Bucket::rbs-discord-alerts-dev gcp:monitoring/alertPolicy:AlertPolicy urn:pulumi:dev::rbs-discord-alerts::gcp:monitoring/alertPolicy:AlertPolicy::rbs-discord-alerts-firestore-queries gcp:storage/bucketObject:BucketObject urn:pulumi:dev::rbs-discord-alerts::gcp:cloudfunctions:CallbackFunction$gcp:storage/bucketObject:BucketObject::rbs-discord-alerts-dev gcp:cloudfunctions/function:Function urn:pulumi:dev::rbs-discord-alerts::gcp:cloudfunctions:CallbackFunction$gcp:cloudfunctions/function:Function::rbs-discord-alerts-dev gcp:cloudfunctions/functionIamMember:FunctionIamMember urn:pulumi:dev::rbs-discord-alerts::gcp:cloudfunctions:CallbackFunction$gcp:cloudfunctions/functionIamMember:FunctionIamMember::rbs-discord-alerts-dev-invoker gcp:monitoring/alertPolicy:AlertPolicy urn:pulumi:dev::rbs-discord-alerts::gcp:monitoring/alertPolicy:AlertPolicy::rbs-discord-alerts-function-executions gcp:monitoring/alertPolicy:AlertPolicy urn:pulumi:dev::rbs-discord-alerts::gcp:monitoring/alertPolicy:AlertPolicy::rbs-discord-alerts-function-error gcp:cloudscheduler/job:Job urn:pulumi:dev::rbs-discord-alerts::gcp:cloudscheduler/job:Job::rbs-discord-alerts-dev gcp:monitoring/dashboard:Dashboard urn:pulumi:dev::rbs-discord-alerts::gcp:monitoring/dashboard:Dashboard::rbs-discord-alerts-dev

Found no pending operations associated with dev

Backend Name pulumi.com URL https://app.pulumi.com/0xJem User 0xJem Organizations 0xJem

Dependencies: NAME VERSION @google-cloud/firestore 6.4.1 @urql/core 3.0.5 cross-fetch 3.1.5 graphql 16.6.0 graphql-tag 2.12.6 @graphql-codegen/cli 2.13.11 @graphql-codegen/typed-document-node 2.3.6 @graphql-codegen/typescript 2.8.1 @graphql-codegen/typescript-operations 2.5.6 @graphql-eslint/eslint-plugin 3.13.0 @graphql-typed-document-node/core 3.1.1 @pulumi/gcp 6.41.0 @pulumi/pulumi 3.46.0 @types/node 14.18.33 @typescript-eslint/eslint-plugin 5.42.0 @typescript-eslint/parser 5.42.0 eslint 8.26.0 eslint-config-prettier 8.5.0 eslint-plugin-import 2.26.0 eslint-plugin-prettier 4.2.1 eslint-plugin-simple-import-sort 8.0.0 eslint-plugin-unused-imports 2.0.0 prettier 2.7.1 ts-node 10.9.1 typescript 4.8.4

Pulumi locates its logs in /var/folders/q6/70msdz2n1s30d81hgj7jjg940000gn/T/ by default

Additional context

I have deployed a project using pulumi recently, and it worked fine. I compared all configurations, etc, and they're the same. The only difference was that I was using secrets in this project.

I removed all usage of requireSecret, replaced it with require and set the variables through pulumi config set (without --secret). I deployed and ran the function, and it worked. So, something to do with requireSecret is tripping it up.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

iwahbe commented 1 year ago

Hi @0xJem, thanks for filing an issue. We'll take a look at this.

In the mean time, you could try eagerly fetching the secret before you deploy:

/**
 * Execution: Google Cloud Functions
 */
const FUNCTION_EXPIRATION_SECONDS = 30;
const discordWebhookURL = pulumiConfig.requireSecret(SECRET_DISCORD_WEBHOOK_URL); // no longer in closure
const functionSubgraphCheck = new gcp.cloudfunctions.HttpCallbackFunction(FUNCTION_NAME_STACK, {
  runtime: "nodejs14",
  timeout: FUNCTION_EXPIRATION_SECONDS,
  availableMemoryMb: 128,
  callback: async (req, res) => {
    console.log("Received callback. Initiating handler.");
    await handler(
      datastore.documentId.get(),
      datastore.collection.get(),
      discordWebhookURL.get(), // Moved to a const above
    );
    // It's not documented in the Pulumi documentation, but the function will timeout if `.end()` is missing.
    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    (<any>res).send("OK").end();
  },
});
0xJem commented 1 year ago

Thanks for the tip! It unfortunately returns an error: Secret outputs cannot be captured by a closure

iwahbe commented 1 year ago

Thanks for the tip! It unfortunately returns an error: Secret outputs cannot be captured by a closure

Thats unhelpful. I'm sorry. You can remove the secretness via pulumi.unsecret.

I'm not an expert on this area, but I think this is true: The function serializer is prompt, so all Output<T> are awaited and evaluated before uploading. The error is to prevent you from leaking a secret value into a lambda by accident. If you want to leak the secret, you need to manually unsecret it first. CC @dixler