[Bug]: Build Environment Variables passed to Dockerfile buildpack

[ChiefORZ]

Description

Heyo!

I created a resource with a "Dockerfile" build pack where i put all of my Environment Variables, which should be included at Build Time with the flag "Build Variable?" checked. These Environment Variables are not being included in the build process, as you can see in the logs below. What am i doing wrong? Do i need to do something else to include these Environment Variables in the build process? Thanks for the help 🙏

#15 [builder 6/8] RUN echo -e "\e[1;33mENVIRONMENT VARIABLES\e[0m"
[2024-Apr-02 08:47:00.447127]
COMMAND:
docker exec h4sgwcc bash -c 'bash /artifacts/build.sh'

OUTPUT:
#15 0.468 -e ENVIRONMENT VARIABLES
[2024-Apr-02 08:47:00.683328]
COMMAND:
docker exec h4sgwcc bash -c 'bash /artifacts/build.sh'

OUTPUT:
#15 DONE 0.5s

#16 [builder 7/8] RUN env
[2024-Apr-02 08:47:00.788315]
COMMAND:
docker exec h4sgwcc bash -c 'bash /artifacts/build.sh'

OUTPUT:
#16 0.260 NODE_VERSION=20.12.0
#16 0.260 YARN_VERSION=1.22.19
#16 0.260 HOME=/root
#16 0.260 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#16 0.260 WIREIT_LOGGER=simple
#16 0.260 PWD=/app
#16 0.260 NODE_ENV=production
[2024-Apr-02 08:47:00.893188]
COMMAND:
docker exec h4sgwcc bash -c 'bash /artifacts/build.sh'

OUTPUT:
#16 DONE 0.3s

Minimal Reproduction (if possible, example repository)

No response

Exception or Error

No response

Version

v4.0.0-beta.248

[ChiefORZ]

even when i try to add them in the Docker like this:

ENV BROWSERLESS_URL=$BROWSERLESS_URL
ENV DATABASE_URL=$DATABASE_URL
ENV GOOGLE_CLIENT_ID=$GOOGLE_CLIENT_ID
ENV GOOGLE_CLIENT_SECRET=$GOOGLE_CLIENT_SECRET
ENV MAGIC_SK=$MAGIC_SK
ENV NEXT_PUBLIC_MAGIC_PK=$NEXT_PUBLIC_MAGIC_PK
ENV NEXT_PUBLIC_UMAMI_SCRIPT_URL=$NEXT_PUBLIC_UMAMI_SCRIPT_URL
ENV NEXT_PUBLIC_UMAMI_WEBSITE_ID=$NEXT_PUBLIC_UMAMI_WEBSITE_ID
ENV NEXTAUTH_SECRET=$NEXTAUTH_SECRET
ENV NEXTAUTH_URL=$NEXTAUTH_URL
ENV REDIS_URL=$REDIS_URL
ENV S3_UPLOAD_BUCKET=$S3_UPLOAD_BUCKET
ENV S3_UPLOAD_KEY=$S3_UPLOAD_KEY
ENV S3_UPLOAD_REGION=$S3_UPLOAD_REGION
ENV S3_UPLOAD_SECRET=$S3_UPLOAD_SECRET
ENV VERCEL_URL=$VERCEL_URL
ENV WEBSNAP_URL=$WEBSNAP_URL

they are still empty and undefined:

#15 [builder 6/8] RUN echo -e "\e[1;33mENVIRONMENT VARIABLES\e[0m"
#15 0.174 -e ENVIRONMENT VARIABLES
#15 DONE 0.2s

#16 [builder 7/8] RUN env
#16 0.268 DATABASE_URL=
#16 0.268 NODE_VERSION=20.12.0
#16 0.268 YARN_VERSION=1.22.19
#16 0.268 NEXTAUTH_SECRET=
#16 0.268 MAGIC_SK=
#16 0.268 HOME=/root
#16 0.268 NEXT_PUBLIC_UMAMI_WEBSITE_ID=
#16 0.268 NEXT_PUBLIC_UMAMI_SCRIPT_URL=
#16 0.268 S3_UPLOAD_BUCKET=
#16 0.268 VERCEL_URL=
#16 0.268 GOOGLE_CLIENT_ID=
#16 0.268 GOOGLE_CLIENT_SECRET=
#16 0.268 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#16 0.268 S3_UPLOAD_REGION=
#16 0.268 S3_UPLOAD_SECRET=
#16 0.268 WIREIT_LOGGER=simple
#16 0.268 NEXTAUTH_URL=
#16 0.268 BROWSERLESS_URL=
#16 0.268 PWD=/app
#16 0.268 WEBSNAP_URL=
#16 0.268 REDIS_URL=
#16 0.268 NODE_ENV=production
#16 0.268 S3_UPLOAD_KEY=
#16 0.268 NEXT_PUBLIC_MAGIC_PK=
#16 DONE 0.4s

[andrasbacsai]

I will add a small fix to the next version.

[ChiefORZ]

@andrasbacsai how is it supposed to work? i updated to 249 and i still do not see the ENV variables:

#15 [builder 6/8] RUN echo -e "\e[1;33mENVIRONMENT VARIABLES\e[0m"
#15 0.239 -e ENVIRONMENT VARIABLES
#15 DONE 0.5s

#16 [builder 7/8] RUN env
#16 0.245 DATABASE_URL=
#16 0.245 NODE_VERSION=20.12.0
#16 0.245 YARN_VERSION=1.22.19
#16 0.245 NEXTAUTH_SECRET=
#16 0.245 MAGIC_SK=
#16 0.245 HOME=/root
#16 0.245 NEXT_PUBLIC_UMAMI_WEBSITE_ID=
#16 0.245 NEXT_PUBLIC_UMAMI_SCRIPT_URL=
#16 0.245 S3_UPLOAD_BUCKET=
#16 0.245 VERCEL_URL=
#16 0.245 GOOGLE_CLIENT_ID=
#16 0.245 GOOGLE_CLIENT_SECRET=
#16 0.245 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#16 0.245 S3_UPLOAD_REGION=
#16 0.245 S3_UPLOAD_SECRET=
#16 0.245 WIREIT_LOGGER=simple
#16 0.245 NEXTAUTH_URL=
#16 0.245 BROWSERLESS_URL=
#16 0.245 PWD=/app
#16 0.245 WEBSNAP_URL=
#16 0.245 REDIS_URL=
#16 0.245 NODE_ENV=production
#16 0.245 S3_UPLOAD_KEY=
#16 0.245 NEXT_PUBLIC_MAGIC_PK=
#16 DONE 0.3s

Should they be defined in the Dockerfile like this?

ENV DATABASE_URL=$DATABASE_URL

[timootten]

@ChiefORZ Did you found a solution for this? :)

[ChiefORZ]

@timootten not yet unfortunately

[timootten]

@ChiefORZ The solution with this: ENV DATABASE_URL=$DATABASE_URL Works for me, in the logs it’s just empty, to not leak the data/key. You have to define all your ENV variables like this and it should work. I had another problem with SvelteKit, their you must use process.env.DATABASE_URL and not the import from SvelteKit

[ChiefORZ]

@andrasbacsai please help, i can not get the environment variables passed to my dockerfile.

here is the setup of my Dockerfile:

FROM node:20 AS nodeenv

ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"

ENV BROWSERLESS_URL=$BROWSERLESS_URL
ENV DATABASE_URL=$DATABASE_URL
ENV GOOGLE_CLIENT_ID=$GOOGLE_CLIENT_ID
ENV GOOGLE_CLIENT_SECRET=$GOOGLE_CLIENT_SECRET
ENV MAGIC_SK=$MAGIC_SK
ENV NEXT_PUBLIC_MAGIC_PK=$NEXT_PUBLIC_MAGIC_PK
ENV NEXT_PUBLIC_UMAMI_SCRIPT_URL=$NEXT_PUBLIC_UMAMI_SCRIPT_URL
ENV NEXT_PUBLIC_UMAMI_WEBSITE_ID=$NEXT_PUBLIC_UMAMI_WEBSITE_ID
ENV NEXTAUTH_SECRET=$NEXTAUTH_SECRET
ENV NEXTAUTH_URL=$NEXTAUTH_URL
ENV REDIS_URL=$REDIS_URL
ENV S3_UPLOAD_BUCKET=$S3_UPLOAD_BUCKET
ENV S3_UPLOAD_KEY=$S3_UPLOAD_KEY
ENV S3_UPLOAD_REGION=$S3_UPLOAD_REGION
ENV S3_UPLOAD_SECRET=$S3_UPLOAD_SECRET
ENV VERCEL_URL=$VERCEL_URL
ENV WEBSNAP_URL=$WEBSNAP_URL
ENV WIREIT_LOGGER="simple"

# Start with the official Node.js image.
FROM nodeenv AS deps
RUN corepack enable

# Set the working directory.
WORKDIR /app

# Copy package.json and pnpm-lock.yaml before other files
# Utilise Docker cache to save re-installing dependencies if unchanged
COPY package.json ./
COPY pnpm-lock.yaml ./

# Install dependencies
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile

# Build the Next.js app
FROM nodeenv AS builder
RUN corepack enable

WORKDIR /app

# Copy all files
COPY . .

# Copy the previously installed dependencies
COPY --from=deps /app/node_modules ./node_modules

# print the env variables
# with a "ENVIRONMENT VARIABLES" title in a different color
RUN echo -e "\e[1;33mENVIRONMENT VARIABLES\e[0m"
RUN env

# Build the Next.js app
ENV NODE_ENV production
RUN pnpm run build:next

# Only copy over the Next.js pieces we need
FROM nodeenv AS runner
RUN corepack enable

ENV NODE_ENV production

WORKDIR /app

COPY --from=builder /app/.next/ ./.next/
COPY --from=builder /app/node_modules/ ./node_modules/
COPY --from=builder /app/prisma/ ./prisma/
COPY --from=builder /app/public/ ./public/

COPY --from=builder /app/src/env.mjs ./src/env.mjs
COPY --from=builder /app/next.config.mjs ./next.config.mjs
COPY --from=builder /app/package.json ./package.json

# Expose the listening port
EXPOSE 3000
ENV PORT 3000

# Run pnpm start to launch the app
CMD ["pnpm", "start:next"]

I marked all my Environment Variables as Build Variables of course. I also recreated the ressource and recreated it completely, because i thought i missed something

[andrasbacsai]

I am trying to reproduce the issue with the same dockerfile (link), but I cannot. It is working with pure dockerfile and with dockerfile from a git repo.

Are you on the latest version (v264) of Coolify?

[ChiefORZ]

@andrasbacsai right, just confirrmed it. it works if i add them manually, but there seems to be an error if i add all of them with the Developer View and change them to Build Variables on the Normal View afterwards

[dmfigueroa]

This is also happening to me. I've spent 5 hours trying to get my environment variables to be read on the Dockerfile bu I haven't been successful yet. I've tried creating them again, locking them, deploying without cache and nothing seems to work. This is my Dockerfile

# syntax = docker/dockerfile:1

# Make sure RUBY_VERSION matches the Ruby version in .ruby-version and Gemfile
ARG RUBY_VERSION=3.3.3
FROM registry.docker.com/library/ruby:$RUBY_VERSION-slim AS base

# Rails app lives here
WORKDIR /rails

# Set production environment
ENV RAILS_ENV="production" \
    BUNDLE_DEPLOYMENT="1" \
    BUNDLE_PATH="/usr/local/bundle" \
    BUNDLE_WITHOUT="development"

ENV RAILS_MASTER_KEY=$RAILS_MASTER_KEY
ENV LITESTREAM_REPLICA_BUCKET=$LITESTREAM_REPLICA_BUCKET
ENV LITESTREAM_ACCESS_KEY_ID=$LITESTREAM_ACCESS_KEY_ID
ENV LITESTREAM_SECRET_ACCESS_KEY=$LITESTREAM_SECRET_ACCESS_KEY
ENV TWITCH_CLIENT_ID=$TWITCH_CLIENT_ID
ENV TWITCH_CLIENT_SECRET=$TWITCH_CLIENT_SECRET

RUN env

# Throw-away build stage to reduce size of final image
FROM base AS build

# Install packages needed to build gems
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential git libvips pkg-config wget

# Install application gems
COPY Gemfile Gemfile.lock ./
RUN bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \
    bundle exec bootsnap precompile --gemfile

# Copy application code
COPY . .

# Precompile bootsnap code for faster boot times
RUN bundle exec bootsnap precompile app/ lib/

# Precompiling assets for production without requiring secret RAILS_MASTER_KEY
RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile

# Final stage for app image
FROM base

# Install packages needed for deployment
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y curl libsqlite3-0 libvips && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Copy built artifacts: gems, application
COPY --from=build /usr/local/bundle /usr/local/bundle
COPY --from=build /rails /rails

# Run and own only the runtime files as a non-root user for security
RUN useradd rails --create-home --shell /bin/bash && \
    chown -R rails:rails db log storage tmp
USER rails:rails

# Entrypoint prepares the database.
ENTRYPOINT ["/rails/bin/docker-entrypoint"]

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD ["bin/bundle", "exec", "foreman", "start"]

This is my output, haven't change a bit no matter what I tried

The empty envs are the ones that Coolify is supposed to add