expo / eas-cli

Fastest way to build, submit, and update iOS and Android apps
https://docs.expo.dev/eas/
MIT License
752 stars 79 forks source link

Running EAS Build with `--local` can't access EAS Secrets but also can't access `.env.local` #2392

Closed melyux closed 3 weeks ago

melyux commented 3 months ago

Build/Submit details page URL

No response

Summary

Cloud EAS Builds have access to EAS Secrets, but local EAS Builds don't. The documentation says the solution is to use environment variables for local EAS Builds instead. However, local EAS Builds only seem to have access to the .env file, but not .env.production nor .env.local nor any other standard .env file resolution that Expo uses.

I tried un-gitignore-ing the .env.local files, and it still didn't work. The logs seem to indicate that it's being loaded:

[PREBUILD] env: load .env.local .env.production .env

But in the end, I still get the Sentry error that the SENTRY_AUTH_TOKEN in .env.local was not loaded: ❌ error: Auth token is required for this request.

If I put SENTRY_AUTH_TOKEN in the plain old .env., the build succeeds.

(P.S. Even if .env.local was in .gitignore, this should still be working, since local builds have no access to EAS Secrets and I don't want to commit my .env.local secrets.)

Managed or bare?

Managed

Environment

expo-env-info 1.2.0 environment info: System: OS: macOS 14.4.1 Shell: 5.9 - /bin/zsh Binaries: Node: 22.1.0 - /opt/homebrew/bin/node npm: 10.7.0 - /opt/homebrew/bin/npm Watchman: 2024.05.06.00 - /opt/homebrew/bin/watchman Managers: CocoaPods: 1.15.2 - /opt/homebrew/bin/pod SDKs: iOS SDK: Platforms: DriverKit 23.5, iOS 17.5, macOS 14.5, tvOS 17.5, visionOS 1.2, watchOS 10.5 IDEs: Xcode: 15.4/15F31d - /usr/bin/xcodebuild npmPackages: expo: ^50.0.0 => 50.0.8 react: 18.2.0 => 18.2.0 react-native: 0.73.4 => 0.73.4 npmGlobalPackages: eas-cli: 9.0.7 Expo Workflow: bare

✔ Check Expo config for common issues ✔ Check package.json for common issues ✔ Check dependencies for packages that should not be installed directly ✔ Check npm/ yarn versions ✔ Check for issues with metro config ✔ Check for common project setup issues ✔ Check for legacy global CLI installed locally ✔ Check that native modules do not use incompatible support packages ✔ Check Expo config (app.json/ app.config.js) schema ✔ Check native tooling versions ✖ Check that packages match versions required by installed Expo SDK ✔ Check that native modules use compatible support package versions for installed Expo SDK

Error output

No response

Reproducible demo or steps to reproduce from a blank project

  1. Create an Expo project with something that requires an environment variable during build, e.g. Sentry
  2. Use an .env.local file in the root with an environment variable needed for the build, e.g. SENTRY_AUTH_TOKEN
  3. Don't .gitignore the .env.local file
  4. Run eas build --platform ios --local and see that it claims to load .env.local variables but they are not made available to parts of the build, e.g. Sentry
curtisgibeaut commented 2 months ago

We are dealing with a similar issue for some reason local builds randomly stopped picking up environment variables.

mgscreativa commented 2 months ago

The same here, put .env and .env.local on project root and using npx eas build --profile production-preview --platform android --clear-cache --local got

[RUN_GRADLEW] env: load .env

But I expected to be .env.local as stated here, here and here

@brentvatne, @dsokal, @szdziedzic can you guys take a look at this?, because it would be great to be able to use .env.local in local eas builds

Environment

expo-env-info 1.2.0 environment info: System: OS: Linux 5.15 Linux Mint 21.3 (Virginia) Shell: 5.1.16 - /bin/bash Binaries: Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node Yarn: 1.22.21 - ~/.nvm/versions/node/v18.19.0/bin/yarn npm: 10.2.3 - ~/.nvm/versions/node/v18.19.0/bin/npm Watchman: 4.9.0 - /usr/bin/watchman npmPackages: expo: ~51.0.14 => 51.0.14 react: 18.2.0 => 18.2.0 react-native: 0.74.2 => 0.74.2 npmGlobalPackages: eas-cli: 10.0.2 Expo Workflow: managed

wongk commented 2 months ago

+1

kevinhylant commented 2 months ago

Just to confirm, when running eas build with the local flag, it should be able to access EAS project secrets & they should be made available in eas.json & app.config.ts via process.env.[variableName], correct? And this behavior would be the same for remote builds too, correct?

I’ve saved my googles-services.json / GoogleService-Info.plist in EAS Secrets for my project & gitingored the files in my repo when I run the build script locally it cannot find the secret files.

I just want to make sure I’m understanding this properly.

thozh commented 1 month ago

I’m encountering a similar issue where updated values in eas.json aren’t being recognized during local builds.

enigosi commented 3 weeks ago

Just to confirm, when running eas build with the local flag, it should be able to access EAS project secrets & they should be made available in eas.json & app.config.ts via process.env.[variableName], correct? And this behavior would be the same for remote builds too, correct?

I’ve saved my googles-services.json / GoogleService-Info.plist in EAS Secrets for my project & gitingored the files in my repo when I run the build script locally it cannot find the secret files.

I just want to make sure I’m understanding this properly.

eas secrets are not supported in local builds https://docs.expo.dev/build-reference/local-builds/#limitations

brentvatne commented 3 weeks ago

The same here, put .env and .env.local on project root and using npx eas build --profile production-preview --platform android --clear-cache --local got

[RUN_GRADLEW] env: load .env

we load env vars specified in your build profile. if you want to also load .env, see more info in this thread: https://expo.canny.io/feature-requests/p/flag-to-load-env-file-during-local-builds

kevinhylant commented 3 weeks ago

@brentvatne thanks for that article/reference, I’m thinking specifically about environment variables like a GOOGLE_SERVICES_JSON which it is recommended be stored as a secret on EAS servers & made available to app.config.ts via process.env.GOOGLE_SERVICES_JSON.

If we save this file to remote EAS Servers & then want to create a local build, how should we access this file in our app.config.ts properly? I’d love to just write .env.process.GOOGLE_SERVICES_JSON like in your docs but I’m not seeing how to do this for local eas builds since it ignores any google_services files I might have in my directory as they’re git ignored.

melyux commented 3 weeks ago

@brentvatne Can you reopen this? As stated in the issue, we can't use the ENV variables in EAS config because they're secrets and the config is committed to git. We can't use .env because that's committed too. We can't use .env.locsl because EAS Build --local doesn't recognize .local env files, unlike Expo itself, which does.

EAS Build --local should recognize .local env files since it has no access to secrets. Right now, the only way to provide secrets to local builds is to literally prefix the secrets to the command like "VAR=something eas build --local ...", which is crazy

szdziedzic commented 3 weeks ago

@kevinhylant The solution that can work here for you is:

  1. Trigger local builds as GOOGLE_SERVICES_BASE64=$(base64 -i ./path/to/google-services.json) eas build --local ... - when triggering the local build the base64 -i ./path/to/google-services.json call will be executed correctly even if the file itself is gitignored.
  2. Add a preinstall hook that will decode GOOGLE_SERVICES_BASE64 to the google-services.json file during the build process:
    {
    "name": "my-app",
    "scripts": {
    "eas-build-pre-install": "echo $GOOGLE_SERVICES_BASE64 | base64 --decode > ./path/to/google-services.json",
    // ...
    },
    "dependencies": {
    "expo": "^51.0.0"
    // ...
    }
    }
szdziedzic commented 3 weeks ago

@melyux I used your repro steps and I was able to reproduce the issue.

However this weird behavior allowing SENTRY_AUTH_TOKEN to be loaded from .env but not from .env.local|development|production|... doesn't come from a bug in EAS build process / @expo/env package (used to load .env files inside of Expo CLI). The values from .env files inside of EAS Build process as of today are only supported inside of actions performed through Expo CLI (npx expo) like prebuild, bundling your JS code and resolving Expo config.

Uploading source maps to Sentry servers (this is where your build fails) is not an action performed through Expo CLI therefore it doesn't have access to .env files through Expo CLI. It turns out that Sentry CLI tries to load the env vars from .env file (and only .env file) by default. If you use .env the Sentry CLI can find the SENTRY_AUTH_TOKEN there, so everything works fine. If you use .env.local it can't, because by default it only checks for .env. That explains the failures that you are observing.

It seems like Sentry CLI supports SENTRY_DOTENV_PATH env variable to choose the .env file to load the env vars from. If you do SENTRY_DOTENV_PATH=../.env.local eas build --local ... your build should work well.

When it comes to making env vars from .env.local accessible throughout the whole build process (in a way that env vars from buildProfile.env are) as of today I would suggest using this simple script inside of the pre-install hook to set environment variables dynamically during the build process using set-env binary.

./source_env_local.sh:

#!/bin/bash

# Path to the .env file
ENV_FILE=".env.local"

# Check if .env file exists
if [ ! -f "$ENV_FILE" ]; then
    echo "Error: $ENV_FILE file does not exist."
    exit 1
fi

# Read .env file line by line
while IFS= read -r line; do
    # Skip empty lines and lines starting with # (comments)
    if [[ -z "$line" || "$line" =~ ^# ]]; then
        continue
    fi

    # Use regex to extract env variable KEY and VALUE
    if [[ "$line" =~ ^([^=]+)=(.*) ]]; then
        key="${BASH_REMATCH[1]}"
        value="${BASH_REMATCH[2]}"

        # Use set-env binary to set the environment variable
        set-env "$key" "$value"
        if [ $? -ne 0 ]; then
            echo "Failed to set $key"
        else
            echo "Set $key"
        fi
    fi
done < "$ENV_FILE"

package.json:

{
  "name": "my-app",
  "scripts": {
    "eas-build-pre-install": "./source_env_local.sh",
    // ...
  },
  "dependencies": {
    "expo": "^51.0.0"
    // ...
  }
}
szdziedzic commented 3 weeks ago

@melyux It's even better to just use direnv to automatically export the env vars form .env when you enter the project directory and then you don't need to worry about running any scripts when doing eas build --local, because env vars are already in your environment

fernandatoledo commented 2 weeks ago

@szdziedzic I tried the option with the script and once again, if I have the envfile tracked and is not in gitignore then it works perfectly but the second I add my envfile to gitignore the script can't find the file 😬

Screenshot 2024-08-07 at 3 20 25 PM

@melyux have you find any solution? Thanks!

szdziedzic commented 2 weeks ago

@fernandatoledo

I tried the option with the script and once again, if I have the envfile tracked and is not in gitignore then it works perfectly but the second I add my envfile to gitignore the script can't find the file 😬

Yeah, that's true eas build --local respects .gitignore by default (if something is gitignored it won't be copied to the eas build --local working directory with the rest of the project, to see how the contents of project archive available during the build process look like use eas build:inspect -p platform --stage archive --output outputPath --profile profileName command). If you want to override the .gitignore behavior and have different ignore rules for eas build --local you can use .easignore.

https://github.com/expo/fyi/blob/main/eas-build-archive.md

I also think that using direnv to export the env vars form .env.local automatically can be a viable option to solve this problem.