Feature Request: Params

[inlined]

We at Firebase don’t always develop in the open. It’s easier to test with a limited audience and then announce it more broadly with a PR splash. Today we’re going to go into depth about our upcoming roadmap, because Cloud Functions for Firebase is between generations of configuration options and the new features don’t seem as full-featured as the old system (yet).

We recently released support for dotenv files and Cloud Secret Manager. Many of you have noticed that, unlike Runtime Config variables, these environment variables are not available when we’re reading your code during deploys. Additionally, Runtime Config variables are not available in Cloud Functions gen 2 environments. We are aware of this feature gap and are working diligently to close it.

Runtime Config: a history

Runtime Config is a strange feature of Cloud Functions for Firebase that exists purely for historic reasons. Runtime Config was originally conceived to allow Cloud Functions for Firebase to dynamically be reconfigured without redeployment. Unfortunately, this flopped during alpha testing: reading the initial state of Runtime Config added about 100ms to cold start latency and any transient outage in Runtime Config would cause uncatchable failures in your Cloud Functions. After discovering this, we immediately removed the “realtime” features of Runtime Config. We kept a dependency on Runtime Config because it served another purpose: it was a way to add “config” values at a time where Cloud Functions did not yet support environment variables.

For configuration at runtime, environment variables and cloud secrets manager are always a better solution. They are standards compliant, keep secrets out of prying eyes, and have emulator support built in. Runtime Config will not exit beta and we’ve been asked to move customers off of it. The removal of Runtime Config support in the firebase-functions SDK and firebase-tools CLI will be announced once their replacement is ready.

Params: the future

Params are an upcoming layer atop of the existing support for environment variables and Cloud Secret manager. They will be very easy to use:

const language = params.defineString(“LANGUAGE”, {
  description: “What language to translate into”
});

export const translate = functions.database.ref(“/messages/{id}”).onCreate(await (ref) => {
  const translated = await translateInto(ref.child(“original”).data(), language.val);
  // …
});

Here, your code depends on the environment variable LANGUAGE at runtime. But this has a few benefits over simple environment variables. The first is that Firebase knows your code needs a “LANGUAGE” environment variable. If we don’t see one, we will prompt you for its value during deploy. This makes it harder to push broken apps to production and makes it easy to get set up with sample code.

The value of a param cannot be read during deploy time, but they can be used to templatize your code:

const bucket = params.defineResource(“BUCKET”, {
  type: “resourceSelector”
  resourceType: “storage.googleapis.com/Bucket”,
});

// bucket.val is the empty string during deploy, but you can assign any attribute of your cloud
// function (e.g. the bucket, your region, min instance counts) and your code will be templatized
export const thumbnail = functions.storage.bucket(bucket).onObjectFinalized((object) => {/*...*/});

We see this as the future because it allows us to provide better features. In the example above, we can provide a dialog that prompts you for a Cloud Storage bucket if your config is missing. The Runtime Config analog would just behave unexpectedly. This feature will make it much easier for developers to create and use templatized sample code or even their own Firebase Extensions. Stay tuned, the future is bright!

Sincerely, The Firebase team

[SrBrahma]

This is interesting. Ain't clear to me if this allows local envs var for dev environment. I use Stripe, and to do local tests, I setup the local Stripe emulator, get its generated secret and set it as a env var to be read in the Functions code, all in a npm script ("STRIPE_DEV_WEBHOOK=$(./stripe listen --print-secret) firebase emulators:start").

[DibyodyutiMondal]

This covers environment variable that we need at run-time. But what if we need a deploy-time environment variable, like NODE_OPTIONS?

Why would I need an environment variable that I don't use at run-time? To specify 'NODE_OPTIONS=--experimental-specifier-resolution=node'. This is so that I don't have to add '.js' to each and every typescript import in the entire dependency tree of the cloud function.

Without that environment variable, there will be a deploy-time error. Because firebase tools will try to load my esm modules and fail. I either need that environment variable, or I need to add '.js' everywhere. And I'd rather use the environment variable.

Again, even if I define params, the typescript files where the params are defined won't be loaded unless the environment has NODE_OPTIONS set in this case.

So far, the dotenv-based solution has allowed me to run the cloud functions with NODE OPTIONS set. But deployment and the emulator both requires monkey patching firebase-tools to include NODE_OPTIONS in the child environment where the triggers are being parsed and executed, respectively.

[inlined]

@SrBrahma Your use case is interesting but should be unaffected by this. CCing @taeold for thoughts on how to allow customers to provide (real) secret values without serializing to the filesystem. Maybe with a commandline flag? We drop environment variables from the emulator so that customers don't accidentally depend on local environment variables and push broken code to prod.

@DibyodyutiMondal ESM is/was an exciting development in the Node and TyepScript landscape. I think this is an interesting use case to consider, but we can't easily allow setting build-time environment variables. They're simply not compatible with our long-term roadmap without careful consideration. My team will take this offline to see if there's something we can do though.

[DibyodyutiMondal]

@inlined

I wouldn't call them build-time variables. My dotenv file contains NODE_OPTIONS and it works in production without any problem. Cloud build has no problems at all. The environment variable is set in the functions. So when they start up, firebase-functions has no problem loading my code, because the env variable is already set.

Very specifically in the context of the problem I'm describing, it is a deploy-time variable, that too on the developer's machine, not the cloud, when firebase-tools is parsing the triggers.

As it currently stands, when parsing the triggers, and while emulating, the firebase-tools library creates a child process, but that process does not inherit environment variables from the parent process. It loads user environment variables after parsing the code. But it can't parse the code without the user environment variables (NODE_OPTIONS), because it can't load the files without the specifiers. Thus it becomes a chicken-or-egg question, and I had to monkey patch so that it could load my code.

In fact, if the child processes I mentioned had the environment variables set before it loaded my code, that would be closer to how the code functioned in the cloud.

EDIT: Unless this behaviour in production is going to change in the new generation. Of which I have little idea.

EDIT: I have many environment variables, but the only one I need to add into the child process is NODE_OPTION, so if the firebase tools gave an option for just that env variable, that would work too. Running the command by passing node options from the command line does not currently work because, as I mentioned, it does not inherit environment variables/options from the parent process. (Which I think is a great, and should not change).

[SrBrahma]

@inlined I am stuck on a older version as env vars aren't supported anymore... I hope you guys can find a way to solve this, so we can set vars on dev env.

[trex-quo]

Is this expected to be released by CFv2 GA? My team has been preparing for our migration and only recently realized that we cannot use CFv2 at all as we cannot deploy functions that rely on environment variables

[inlined]

This will be part of Cloud Functions for Firebase v2 GA but is not a blocker for public preview. We are considering multiple avenues, including build env files.

[elhe26]

@inlined just to be on the same page, can I use params without cloud secret manager?

[inlined]

@elhe26 For non-sensitive params we back everything in dotenv files. For secret params we use CSM exclusively as our backing store at this time.

[elhe26]

@elhe26 For non-sensitive params we back everything in dotenv files. For secret params we use CSM exclusively as our backing store at this time.

@inlined,

What I'm referring to is the ability to use params at runtime (when deploying functions) without using cloud secret manager. We had to rollback to a previous version where runtime config is working.

[inlined]

I'm overjoyed to announce that params are included in yesterday's release of 4.0.0!

Here's an (overly) parameterized version of our uppercase sample:

import { defineString, defineBoolean, gcloudProject } from "firebase-functions/params";
import { onValueCreated } from "firebase-functions/v2/database";

const collection = defineString("COLLECTION", {
  label: "collection",
  description: "The collection of documents to uppercase",
});

const originalField = defineString("ORIGINAL_FIELD", {
  label: "originalField",
  description: "The field of the message to uppercase",
  default: "message",
});

const targetField = defineString("TARGET_FIELD", {
  label: "targetField",
  description: "The target field to put the uppercased message",
  default: "uppercase",
});

const isProd = defineBoolean("IS_PROD", {
  label: "isProductionProject",
  description: "Should this project be initialized with production resources allocation",
  default: gcloudProject.equals("my-prod-project"),
})

export const uppercase = onValueCreated({
  ref: `${collection}/{id}`, 
  minInstances: isProd.thenElse(1, 0),
}, async (event) => {
  const original: string = event.data.val()[originalField.value()];
  await event.data.ref.child(targetField.value()).set(original.toUpperCase());
});

This creates the following prompts in the deploy experience:

  functions: preparing codebase default for deployment
? Enter a string value for collection: 
(The collection of documents to uppercase) messages
? Enter a string value for originalField: 
(The field of the message to uppercase) message
? Enter a string value for targetField: 
(The target field to put the uppercased message) uppercase
? Enter a boolean value for isProductionProject: 
(Should this project be initialized with production resources allocation) true

You can use a parameter directly at configuration time (see the minInstances config) or even with config string interpolation (see the ref config). This will templatize your deployment. To get the actual value at runtime, call .value() on the param.

This feature not only brings back parameterized configuration, but better documents your code. You can now write self-documenting OSS examples and/or ensure that you never deploy to a project that's missing part of the environment configuration.

[LaCocoRoco]

@inlined Could you please explain how this should fix the empty environment variables on deploy? Based on your reference it should be used like this:

import { initializeApp } from 'firebase-admin';
import { cert } from 'firebase-admin/app';
import { https } from 'firebase-functions';

const projectId = defineString('CREDENTIALS_PROJECT_ID', {
  label: 'projectId',
});

const privateKey = defineString('CREDENTIALS_PRIVATE_KEY', {
  label: 'privateKey',
});

const clientEmail = defineString('CREDENTIALS_CLIENT_EMAIL', {
  label: 'clientEmail',
});

initializeApp({
  credential: cert({
    projectId: projectId.value(),
    privateKey: privateKey.value(),
    clientEmail: clientEmail.value(),
  }),
});

export const httpsOnRequest = https.onRequest(async (req, res) => {
  res.status(200);
});

But the deploy fails because the environment variables are still empty on deploy.

[wian-plus]

As an example to the above where we'd like to use params for deployment:

const isProd = defineBoolean("IS_PROD", {
    label: "isProductionProject",
    description: "Should this project be initialized with production resources allocation",
    default: false,
});

const secondApp =
    initializeApp(
        {
            databaseURL: isProd.value() ? 'prod-url' : 'dev-url'
            storageBucket: "gs://something.com",
        }, "secondApp")

const thirdApp =
    initializeApp(
        {
            databaseURL: isProd.value() ? 'prod-url' : 'dev-url',
            storageBucket: "gs://something.com,
        }, "thirdApp")

export const secondAppFirestore = getFirestore(secondApp)
export const thirdAppDatabase = getDatabase(thirdApp)

This gives the warning (during deploy or emulation):

{"severity":"WARNING","message":"params.IS_PROD.value() invoked during function deployment, instead of during runtime."}
{"severity":"WARNING","message":"This is usually a mistake. In configs, use Params directly without calling .value()."}
{"severity":"WARNING","message":"example: { memory: memoryParam } not { memory: memoryParam.value() }"}

How can we achieve the above using params?

[ishowta]

If I'm not mistaken, here's what I understand

  | Parameterized configuration | Environment variables | secrets | Environment configuration (runtimeconfig) -- | -- | -- | -- | -- set value | dotenv or Cloud Secret Manager with `defineString("FOO")` or `defineSecret("SECRET_NAME")` | dotenv | firebase functions:secrets:set | firebase functions:config:set using value on deploy-time | ✕ ([why](https://github.com/firebase/firebase-functions/issues/1244#issuecomment-1266059434)) | ✕ ([why](https://github.com/firebase/firebase-functions/issues/1244#issuecomment-1266059434)) | ✕ | `functions.config().foo.bar` using value on deploy-time in `runWith` params | `functions.runWith({ minInstance: foo})` (foo variable is stringified to ‘foo’ and get value on functions internal) | ✕ | ✕ | `functions.config().foo.bar` using value on runtime | `foo.value()` | `process.env.FOO` | `process.env.SECRET_NAME` | `functions.config().foo.bar` is secure | ○ (using GCP Secret Manager if you use `defineSecret`) | △ (dotenv) | ○ (using Cloud Secret Manager) | △ (using Cloud Runtime Configurator) v2 | ○ |   |   | ✕ status |   |   |   | deprecated ([why](https://github.com/firebase/firebase-functions/issues/1244#issuecomment-1266059434))

[wian-plus]

This neither works for database instance config:

const isProd = defineBoolean("IS_PROD", {
  label: "isProductionProject",
  description: "Should this project be initialized with production resources allocation",
  default: gcloudProject.equals("my-prod-project"),
});

export const realtimeDbListener = region('europe-west1')
.runWith(
    {
        minInstances: isProd.thenElse(1, 0) // This works fine
    }
)
.database
.instance( isProd.thenElse("prod-db-id", "dev-db-id") ) // This does not accept the config
.ref("your-path")
.onUpdate(async (snapshot, context) => {

 });

[knownasilya]

Does this feature support reading from .env.local when using the emulator?

[rohitgarg-lenskart]

Currently we can't access custom parameters while deployment of functions or while serving functions locally. Only a few selected params are allowed. e.g. minInstance Can we expect custom parameters to be added, to access while deployment or while serving. Unlike, this issue https://github.com/firebase/firebase-tools/issues/5524, where few of the values are only added to params.

Even, the most basic thing used by functions "region" is still not added, and functions.config() is deprecated, just to make things tough for the users?

If any alternatives to access custom parameters while deployment, please provide.

Use Case 1: Need to select different regions to deploy functions based on parameters. Use Case 2: Need to select different environment and a lot of pre-deployment conditions are dependent upon the environment being selected. eg.functions.storage.bucket("bucket-name").object().onFinalize, here bucket name can be different for different projects, and the bucket name has to given while deployment itself,

[mustafaekim]

If a node_module relies on an environmental variable, how can I ensure that the library configures itself correctly upon importing the module? The module is imported within the index.ts file located in the functions/src directory.