withastro / astro

The web framework for content-driven websites. ā­ļø Star to support our work!
https://astro.build
Other
45.58k stars 2.39k forks source link

Secret env variables undefined during server build with Cloudflare SSR #4416

Closed rodneylab closed 2 years ago

rodneylab commented 2 years ago

What version of astro are you using?

1.0.6

Are you using an SSR adapter? If so, which one?

Cloudflare

What package manager are you using?

pnpm

What operating system are you using?

Mac

Describe the Bug

Non-public environment variable is not defined during Cloudflare server build, though it works fine in local build. I created a minimal reproduction with test to demonstrate this.

Test

I read env variables in API routes:

// src/pages/api.ts
import type { APIRoute } from 'astro';

export const get: APIRoute = async function get() {
  const secret = import.meta.env.SECRET;
  const text = secret ? secret.length.toString(10) : "-1"; 
  return new Response(text, {
    status: 200,
  });
};

export const post: APIRoute = async function post({ request }) {
  const { secret } = await request.json();
  const text = secret ? secret.length.toString(10) : "-1";
  return new Response(text, {
        headers: { 'content-type': 'application/json' },
    status: 200,
  });
};
// src/pages/api-public.ts
import type { APIRoute } from 'astro';

export const get: APIRoute = async function get() {
const secret = import.meta.env.PUBLIC_SECRET;
    return new Response(secret, {
        status: 200,
    });
}

Then output them on page (for testing purposes):

---
const { url } = Astro;
const response = await fetch(`${url}api`);
const secret = await response.text();

const publicResponse = await fetch(`${url}api-public`);
const publicSecret = await publicResponse.text();

const serverSecret = import.meta.env.SECRET;
const postResponse = await fetch(`${url}api`, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ secret: serverSecret })
});
const postSecret = await postResponse.text();
---
<!-- TRUNCATED -->
    <p>{JSON.stringify({secret, publicSecret, postSecret})}</p>
<!-- TRUNCATED -->

Local build

run pnpm build and pnpm wrangler pages dev ./dist and see expected result:

Screenshot 2022-08-22 at 07 15 32

Although the site builds fine, the CLI outputs the following error:

šŸš§ 'wrangler pages <command>' is a beta command. Please report any issues to https://github.com/cloudflare/wrangler2/issues/new/choose
āœ˜ [ERROR] _worker.js is importing from another file. This will throw an error if deployed.

Cloudflare server build

Added the environment variables in the Cloudflare Pages console:

Screenshot 2022-08-22 at 06 33 04

Build itself runs OK with no errors output, but unlike local build, it looks like the secret is not available. That is neither when directly used in the function (get function above in api.ts), nor in the frontmatter of the Astro file (put function). PUBLIC_ prefixed variable works as expected:

Screenshot 2022-08-22 at 07 20 23

I expect the length of the secret variable to be output form this Cloudflare server build, just like on the local build.

I need to be able to access a private key in the API function. Is there a workaround other than making it PUBLIC_?

Extra detail

Examining /dist/_worker.js locally after build I can see the variables are present:

Ce=Ca(async(t,a,i)=>{let n=t.createAstro(Mi,a,i);n.self=Ce;let{url:o}=n,e=await(await fetch(`${o}api`)).text(),p=await(await fetch(`${o}api-public`)).text(),c="my-secret",l=await(await fetch(`${o}api`,{method:"POST",headers:{"content-type":"application/json"},body:JSON.stringify({secret:c})})).text()

Local versions

Node: v16.16.0
Wrangler: 2.0.26
pnpm : 7.9.3

I set Node version to 16 for server build using .nvmrc file.

Link to Minimal Reproducible Example

https://github.com/rodneylab/astro-cloudflare

Participation

matthewp commented 2 years ago

Things I'd like to know:

bluwy commented 2 years ago

Is our private env handling build time or runtime?

It looks like it's build-time

I believe this is because Astro's env handling looks for a .env file and builds with it. Since the env vars on cloudflare are in process.env.* only, it isn't included in the build. A workaround is to have a custom pre-build script that reads the process.env.* and write into .env.

Perhaps we can try reading the process.env.* too in the build process. I'll check if this is possible.

bluwy commented 2 years ago

Actually we do read from process.env.* but the problem is because process.env. doesn't exist in Cloudflare workers. What Astro does under the hood is that it transforms import.meta.env.SECRET into process.env.SECRET, which wouldn't work.

A workaround for now is to use vite.define to replace them yourselves, e.g.

export default defineConfig({
  vite: {
    define: {
      'process.env.SECRET': JSON.stringify(process.env.SECRET)
    }
  }
})

I don't think there's much we can do for this issue, other than documenting this, and probably re-work env handling in the future to support Cloudflare (?).

rodneylab commented 2 years ago

Thanks, that’s working fine šŸ˜

maxcountryman commented 1 year ago

@bluwy that workaround is not working for me. Did something change recently?

cauboy commented 1 year ago

I just ran into the same issue. I was following Astro's documentation for using environment variables with the cloudflare adapter (https://docs.astro.build/en/guides/integrations-guide/cloudflare/#environment-variables) but environment variables weren't defined when deployed to Cloudflare Pages. The issue doesn't pop up locally when running astro dev or when simulating Cloudflare pages better by running astro build && wrangler pages dev ./dist.

// pages/index.astro
---
const apiBaseUrl = import.meta.env.API_BASE_URL
---

<div>{apiBaseUrl}</div>

Only after adding the following I was able to access the env variables in Cloudflare pages

export default defineConfig({
  vite: {
    define: {
      'process.env.API_BASE_URL': JSON.stringify(process.env.API_BASE_URL)
    }
  }

Now, I'm unsure, whether the whole thing is a bug, or if it custom env variables should always be defined that way when using the cloudflare adapter (which would mean the docs are misleading)

lucaswalter commented 1 year ago

Hello, I have been fighting this exact same issue still and have resorted to the vite: { define: {} } solution. Is this still how we are supposed to access Cloudflare env vars? Or is another way supposed to work?

bluwy commented 1 year ago

I'd suggest creating a new issue with a repro as we don't usually track long-closed issues.

niklauslee commented 5 months ago

I finally fixed by:

  1. delete .env
  2. add env vars in vite.define in astro.config.mjs
  3. add env vars in cloudlfare settings
dcastillogi commented 2 months ago

Problem Description: I was unable to read environment variables despite configuring them in the Cloudflare dashboard.

Solution: I resolved the issue by modifying my Astro configuration as follows (see vite config):

export default defineConfig({
  integrations: [tailwind(), react()],
  output: "hybrid",
  adapter: cloudflare(),
  vite: {
    define: {
        "process.env": process.env
    }
  }
});

This change allows the environment variables to be accessed correctly and everything works smoothly now. They can be read as usual: import.meta.env.VARIABLE_NAME.

thebrownfox commented 2 months ago

Problem Description: I was unable to read environment variables despite configuring them in the Cloudflare dashboard.

Solution: I resolved the issue by modifying my Astro configuration as follows (see vite config):

export default defineConfig({
  integrations: [tailwind(), react()],
  output: "hybrid",
  adapter: cloudflare(),
  vite: {
    define: {
        "process.env": process.env
    }
  }
});

This change allows the environment variables to be accessed correctly and everything works smoothly now. They can be read as usual: import.meta.env.VARIABLE_NAME.

^ This could be added to docs.

lukeojones commented 2 months ago

For anyone looking at this issue and still having no luck after adding the extra vite config...

I've twice now experienced an issue where the secrets/env-vars needed to be added to the worker via the dashboard prior to deployment. I had initially been using wrangler CLI to do this but only had success with the above solution once the env vars had been added to my worker settings (in the cloudflare dashboard) prior to running the deployment.

I can't explain why this makes a difference but it has solved the issue twice now (on totally different projects)