Docs: .env and .env.<stage> usage

[garysassano]

Implement env variables loading with modes, similarly to Vite.

Include dotenv-expand to allow reusing env vars within the same file:

BASE_URL=https://example.com
API_ENDPOINT=$BASE_URL/api/v1

Optionally, use Node 20.6.0+ native feature for dotenv.

[thdxr]

so we currently will load .env and .env.<stage> is that what you are looking for or something more?

[garysassano]

Where is that behaviour documented? I couldn't see it mentioned under secret.

[paolostyle]

Not sure if this should be a separate issue but I'm having trouble using secrets loaded through the .env file. I actually explicitly loaded it with sst secret load .env and that worked fine, but then when I'm trying to use it in the config like this:

    api.route('ANY /{proxy+}', {
      handler: 'src/index.handler',
      environment: {
        DB_CREDENTIALS_SECRET: new sst.Secret('DB_CREDENTIALS_SECRET').value,
      },
    });

I'm getting this error when attempting to deploy:

Invalid component name "DB_CREDENTIALS_SECRET". Component names must start with an uppercase letter and contain only alphanumeric characters.

I find that limitation a bit strange in general but in this particular case it's quite strange as this is the generally accepted convention for .env variables. I'd expect this to work or the sst secret load should "translate" it to the correct format.

[garysassano]

Shouldn't you use sst secret set for having a secret available within a function?

[paolostyle]

From what I understand sst secret load is basically batch secret set.

[jayair]

@paolostyle Yeah a couple of things, you probably don't want to pass the secret value into the environment like that. Instead link the route to the secret.

should "translate" it to the correct format

Yeah hmm I don't know if we should do it automatically because then it wouldn't be obvious what the secret name is. On the other hand, we should probably not let you set a secret with the incorrect format. I'll open an issue for that. https://github.com/sst/ion/issues/515

[jayair]

Where is that behaviour documented? I couldn't see it mentioned under secret.

Yeah we should document this. I'll change this issue to that and assign it.

[garysassano]

Btw, I saw other people that did stuff like this:

package.json

"deploy-dev": "export ENV=development && export AWS_DEFAULT_PROFILE=default && sst deploy",
"deploy-prod": "export ENV=production && export AWS_DEFAULT_PROFILE=default && sst deploy --stage=production",

sst.config.ts

/// <reference path="./.sst/platform/config.d.ts" />

import { config } from "dotenv";

const env = process.env.ENV ?? "development";

const prodCapital = env === "production" ? "P" : "D";

export default $config({
  app(input) {
    return {
      name: "sst-aws-nextjs-test",
      removal: input?.stage === "production" ? "retain" : "remove",
      home: "aws",
    };
  },
  async run() {
    const confObject = config({
      path: `.env.${env}`,
    }).parsed;

    const serverEnvs: any = {
      ...confObject,
      HELLO: "WORLD",
    };

    const site = new sst.aws.Nextjs(`Web${prodCapital}`, {
      environment: serverEnvs,
    });
  },
});

All this mess just because they weren't aware of the sst secret. They simply wanted to have the stage-related (development or production) environment variables available within their Next.js function.

Honestly, I have good reason to believe this is an extremely common use case, so maybe SST could step forward and make this even easier for the end user.

What I mean is that, when calling sst deploy --stage=production, SST could automatically do the following:

• Bundle an environment variable SST_STAGE=production
• Bundle all environment variables from .env.production if present

Similarly, if you were to call sst deploy --stage=development:

• Bundle an environment variable SST_STAGE=development
• Bundle all environment variables from .env.development if present

If someone doesn't like this automatic bundling of environment variables to make them available to your application, you could turn this feature into a parameter which you can optionally disable.

[jayair]

It sounds like from what Dax is saying it already loads .env.. And there's the $app.stage variable.

Is there anything else you need?

[garysassano]

Is there anything else you need?

It would be nice to have the automatic bundling of env variables within your site, the feature I mentioned on my message above.

[jayair]

You mean automatically loading the .env into your site? I'm sure if we can do that automatically because you might have multiple sites in your app.

[dmeehan1968]

Loading of env vars from .env and .env. doesn't appear to be working in 0.0.399 when deploying, but does work when in dev mode.

When starting in dev mode, I see:

 ▲ Next.js 14.2.3
  - Local:        http://localhost:3000
  - Environments: .env.development, .env

And the vars listed in those files are loaded into process.env.

When I deploy, I see:

  ▲ Next.js 14.2.3
  - Environments: .env.production, .env
   Creating an optimized production build ...

But the vars DO NOT end up in the environment. I can verify this by putting a console.log(process.env) in my middleware file, and see that in local dev the values are set, and in deployed production none of them are set (regardless of whether they are vars from .env or .env.production).

Observations:

1. The load order appears to be contrary to what would be expected, as generally .env should be for all stage vars, and .env. should be overrides, but at the moment I seem to get the value from .env instead of the more specific (when in local dev mode) but that's consistent with the quoted load order above.
2. .env.<stage> doesn't seem to be quite the right terminology, as the sst commands show the stage to be (unless otherwise specified) named after my AWS user id. Yet the build process does pick up .env.development in local dev mode, and .env.production for the deploy.
3. The comments in this issue seem to be conflating 'secrets' and 'env vars', as these are distinct things, even if the environment is used to carry them to the appropriate runtime. Env vars is a reasonable way to pass private credentials to the runtime - secret does this itself, just passing the values as SST_RESOURCE_* env vars. The advantage to Secrets is that they are type safe as they end up with definitions in sst-env.d.ts and can be referenced by import { Resource } from 'sst'.
4. I note that there is another issue for AWS Secrets Manager support (#221) which would actually be a more secure way for those that need it (logging process.env can cause secrets to leak - granted any logging of secrets will do that, but by lumping them in with other config relates vars increases the risk of unintentional logging). AWS SM does have the downside of a cost overhead for storage and retrieval.

[jayair]

Just to be clear, the .env loading that we were talking about is related to your sst config. Not in your Next.js app. Can you verify that they are being loaded in your sst config in dev and deploy?

[dmeehan1968]

@jayair Aah, ok.

• .env gets loaded for both dev and deploy
• .env.development does not get loaded for dev
• .env.production does not get loaded for deploy

Based on what I see when I console.log(process.env) from sst.config.ts. Note as above that sst dev/deploy SAYS that it is using the those files as appropriate, but none of the vars get passed. It's now clear to me that dev mode only gets the vars from .env by virtue of being in the same shell, not because they are actually passed to the NextJs instance. This also explains why the .env.development vars don't override .env, because they aren't loaded at all.

Note that my NextJs install is in a subpath from the sst root, so I have

sst.config.ts
packages/dashboard/.env
packages/dashboard/.env.development
packages/dashboard/.env.production

My expectation here is that .env* files in the NextJs directory should be loaded according to stage context, AND passed to the NextJs instance/function so that they are available as expected. Am I wrong to have this expectation?

Does this mean that within sst.config.ts I have to manually pass the relevant variables to NextJs/Function via its 'environment' prop? (Which means remembering to add them both to the .env* file and pick them from process.env in sst.config.ts, which seems error prone)

Is there a naming convention that assists this?

[jayair]

Yeah .env is loaded only to the sst.config.ts. You can then read from that and pass it using the environment to the function or site you want. The reason for not automatically doing this is because you can have multiple of these targets and you most likely don't want to load everything everywhere.

I haven't personally tested the .env.<stage> parts. If there are some issues with this let me know and we can take a look.