Improved env handling

[florian-lefebvre]

• Accepted Date: 2024-02-09
• Reference Issues/Discussions: #822
• Author: @florian-lefebvre
• Champion(s): @florian-lefebvre (with some help!)
• Implementation PR: N/A

Summary

This RFC aims to improve DX (and eventually security) around env variables.

Background & Motivation

Env variables are an important part of any web application. You need to store sensitive data (think API secrets, tokens etc) without being leaked inside your git repo. But that's only the 1st part of the story. It's easy to leak this data by importing in the wrong place, eg. the frontend like Resend a few weeks ago.

Other JS frameworks (eg. SvelteKit) are handling env pretty well. From my understanding, the env story is currently a bit tricky in Astro. According to the docs, here is how env variables are currently handled:

• Astro uses Vite’s built-in support for environment variables
• Static variables (ie. replaced statically at build time) are accessible via import.meta.env
• import.meta.env includes some default variables like SSR, BASE_URL...
• A public variable key has to be prefixed by PUBLIC_
• A non public variable accessed using import.meta.env on the client side will be undefined (value will be accessible server side)
• Env variables can be loaded through .env (or .env.production, .env.development) and CLI
• Any non built-in variable can be manually typed
• Runtime variables should be access using process.env, or following the used runtime (eg. Deno.env.get() for the deno adapter)
• I'm not sure if process.env has any protection against client-side usage (like import.meta.env), but I guess it doesn't

Goals

• Provide a fully type-safe experience for environment variables, without manual type definitions
• Allow users to mark environment variables as required, providing hints when a variable is missing
• Reduce user confusion between inlined, static variables and dynamic, runtime variables
• Allow adapters to specify how runtime env variables should be handled

Non-Goals

• Complex validation or type casting of env variables. We might want to enable this at some point, but there is likely a performance cost for this at runtime. We should punt on this if possible!
• Allowing integrations to define environment variable constraints. This is a great idea, but until we know what this feature looks like, we should consider it out of scope.
• Future: allow adapters to customize build-time variable handling. Better runtime handling is the most important problem this proposal aims to solve, but if we find an API that enables this as well, that's great.

[bluwy]

1. buildTime and runtime reads weird to me 😄 Personally I prefer static/dynamic but not a hill I'd die on.
2. I think overriding constants is fine as it's only an implementation detail.
3. How would we support public runtime variables though? For example, how would PUBLIC_FOO be referenced in runtime? If the variable is already public, they can switch to buildtime instead.

[florian-lefebvre]

buildTime and runtime reads weird to me 😄 Personally I prefer static/dynamic but not a hill I'd die on.

I'm comfortable with both namings tbh but idk what's the best way to gather feedback as some kind of poll on gh ngl

How would we support public runtime variables though? For example, how would PUBLIC_FOO be referenced in runtime? If the variable is already public, they can switch to buildtime instead.

That wouldn't change anything on the server side, but it would require extending the window with the public runtime values and use those in the client side version of the virtual import. This adds some little JS to every page and tbh, I don't see the benefit much either

[ematipico]

About the naming, I think we should reach out to docs for an opinion.

We keep naming things based on what are internally, but we have to consider how docs teach concepts to our users. For example they try to push the term "on demand" instead of dynamic/SSR.

To echo what @bluwy said, naming is hard and subjective, so I would feel more comfortable to use any name that docs suggests, because at the end they will teach the narrative to our users.

[florian-lefebvre]

A few comments from @FredKSchott on Discord

Fred

SST calls these "Secrets" (runtime) vs. "Parameters" (static) which I always liked, although it implies that runtime things are always secret which may not always be neccessarily true.

However, if the big thing you need to know about static is that they get bundled into your build itself, then a name should definitely make that distinction clear

aka the distinction I would care about is that it's not really about runtime vs. buildtime, but more about "included in your build = insecure" vs. "read from the server at runtime = secure" (which gets back to why I think "Secrets vs. Parameter" is actually pretty good, all things considered)

no strong objections to the other names though, at the end of the day. Static vs. Dynamic is pretty good as well.

Florian

the thing is, "included in your build" is not necessarily unsecure. I mean, if you prefix it with PUBLIC_ then it means it's safe to use client side right?

Fred

Anything in the deployed code (not read from ENV) should be considered insecure as a default, yes. For ex:

• Deploying it to your own machine = the secret is readable to any user account with SSH + file read access
• Deploying it to Vercel = the secret is readable from the "source" tab (possibly Netlify, CF, etc. as well?)

[bholmesdev]

That's some good perspective! I notice Cloudflare and GitHub also use "secret" vs "variable" (or parameter in SST's case). It seems like we're not agreed on whether buildtime vs. runtime is synonymous with variable vs. secret. I think static vs. dynamic would be more agreeable thinking it over

[alexanderniebuhr]

The only issue I have with static vs. dynamic is, that those variables are not really "static" or "dynamic". Both types can be changed/overriden by the user in the runtime. Both types can be updated with a config update (for some hosts that needs a redeployment for both types). I associate a changing value as "dynamic", e.g. timestamp

[klizter]

Built in support for dynamic PUBLIC environment variables would be great. We just recently ran into the issue of getting the same build to work in several different environments, because public variables are baked in at build time. I hope this will work in the future, for now we have to handle this issue outside Astro.

[bluwy]

@klizter If we support dynamic public env vars, how would you envision the API to pass in the different env var value dynamically? (specifically client-side / browsers). On the server-side, it's simpler with process.env or Deno.env, but on the client-side, it's not clear to me how that's being passed. Hence, I suggested above to not support it.

[florian-lefebvre]

Stage 3 RFC available in #894

[nemanjam]

How do I access env variables in the astro.config.ts itself? Can they be defined and used in the same file? For example i store site url as env variable site: SITE_URL.

import { envSchema } from './src/utils/env';
const { SITE_URL } = CONFIG;

export default defineConfig({
  site: SITE_URL, // I still must use process.env.SITE_URL here?
  experimental: {
    env: {
      schema: envSchema,
    }
  },
...
});

[florian-lefebvre]

Hey @nemanjam thanks for bringing this up! It's out of scope of this RFC but we're not against it! That's something often asked and source of confusion so I think that's something we could tackle. Would you mind creating a new roadmap discussion for this with as much info/context as you can? Then we can share it to gather more feedback and usecases