continuouspipe / dockerfiles

Docker base images for many technologies.
MIT License
46 stars 28 forks source link

ENV variables not being set before composer install on image PHP-NGINX #334

Closed richdynamix closed 7 years ago

richdynamix commented 7 years ago

I don't think I am doing anything wrong, just what I am used to doing but anything I set in environment of docker-compose.yml is not being picked up when I have a composer project.

For example. I am setting DEVELOPMENT_MODE and GITHUB_TOKEN but these aren't being picked up when the container is being built.

docker-composer

version: "2"
services:
    web:
        build:
            context: .
            args:
                - APP_ENV
        links:
            - database
            - redis
        expose:
            - 80
            - 443
        ports:
            - "80:80"
            - "443:443"
        volumes:
            - .:/app
        environment:
            VIRTUAL_HOST: "127.0.0.1:80"
            REDIS_HOST_PORT: 6379
            APP_USER_LOCAL: "true"
            GITHUB_TOKEN: "XXXXXXXXXXXXXXXXX"
            DEVELOPMENT_MODE: "true"

    database:
        image: quay.io/continuouspipe/mysql5.7:stable
        environment:
            MYSQL_ROOT_PASSWORD: project
            MYSQL_DATABASE: project
            MYSQL_USER: project
            MYSQL_PASSWORD: project
        expose:
            - 3306
        ports:
            - "3306:3306"

    redis:
        image: quay.io/continuouspipe/redis3:stable
        expose:
            - 6379
        ports:
            - "6379:6379"

Dockerfile -

FROM quay.io/continuouspipe/php7.1-nginx:stable

ARG APP_ENV=

COPY ./tools/docker/etc/ /etc/
COPY ./tools/docker/usr/ /usr/

COPY . /app
WORKDIR /app

RUN container build

When I run this I get the following during composer install -

+ run_composer
+ '[' -n '' ']'
+ as_code_owner 'composer install --no-interaction --optimize-autoloader'
+ set +x
+ sudo -u build -E HOME=/home/build /bin/bash -c 'cd '\''/app'\''; composer install --no-interaction --optimize-autoloader'
Loading composer repositories with package information

Which illustrates that the DEVELOPMENT_MODE and GITHUB_TOKEN are not read as the composer flags are not applied and the GITHUB_TOKEN is missing from the condition.

If I remove the composer.json file from the project and just build the container with no install it starts fine and I can echo out the variables in the container -

root@ecaa020210ce:/app# echo $DEVELOPMENT_MODE
true

Strangely enough, the .env file that I have setup to be generated with the confD templates is not generated unless the composer.json file is missing.

It must be a configuration issue on my side but I can't see what could cause this. Any ideas would be greatly appreciated.

kierenevans commented 7 years ago

Hey @richdynamix,

To affect variables during the build stage, you'll need to pass GITHUB_TOKEN and DEVELOPMENT_MODE in as ARGs in the Dockerfile, with appropriate changes to the docker-compose.yml (and continuous-pipe.yml if being used) to pass the variables through in the build section.

Note that this will make the token part of the history of the image (fetchable with docker history --no-trunc <image>), so please ensure you are pushing to a private docker repository.

I don't think that confd will be running during the build of the image unless you were to override a function to call it, for example:

alias_function do_build do_project_build_inner
function do_build() {
  do_templating
  do_project_build_inner
}

in your plan.sh file.

I hope this helps!

richdynamix commented 7 years ago

Ah I never had the templating function in my build. I can see that is running but same outcome :(

Dockerfile

FROM quay.io/continuouspipe/php7.1-nginx:stable

ARG APP_ENV=
ARG GITHUB_TOKEN=
ARG DEVELOPMENT_MODE=

COPY ./tools/docker/etc/ /etc/
COPY ./tools/docker/usr/ /usr/

COPY . /app
WORKDIR /app

RUN container build

plan.sh

alias_function do_build do_build_project_inner
do_build() {
  do_templating
  do_build_project
}

I can see the following output but no token or flags -

+ '[' build == start ']'
+ set -- build
+ shift 1
+ do_build
+ do_templating
+ do_confd
+ confd -onetime -backend env
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Backend set to env
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Starting confd
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Backend nodes set to
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /app/.env out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /app/.env has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO /etc/nginx/nginx.conf has md5sum 907bbf7d1cb3f410d8d6d4474a984b86 should be d78f00d1bd733acd9ed310a42cb76d9a
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/nginx.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/nginx.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_basic_auth.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_basic_auth.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/htpasswd out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/htpasswd has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_ip_whitelist.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_ip_whitelist.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_log_format.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_log_format.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_realip.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_realip.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_server_name_bucket_size.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/conf.d/nginx_server_name_bucket_size.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO /etc/nginx/sites-available/default has md5sum f9085fae559334aba3e6409d717131e6 should be 04d699244287221b9d82455d0d91ddaf
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-10-base.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-10-base.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-05-custom_scheme_flags.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-05-custom_scheme_flags.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-05-site_healthcheck.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-05-site_healthcheck.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-30-phpfpm.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-30-phpfpm.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-06-site_redirect_to_https.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-06-site_redirect_to_https.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-20-rewriteapp.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/nginx/sites-available/default-20-rewriteapp.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO /etc/php/7.1/fpm/pool.d/www.conf has md5sum 385899c6bb717a8b16e4dad2544b117e should be 9cf301692287ccafc8913ee339d03af0
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/fpm/pool.d/www.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/fpm/pool.d/www.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO /etc/php/7.1/fpm/php-fpm.conf has md5sum c4186e0e75f0a461887e1083c1d4f939 should be 6e0aa1a034986bc841c491296a19ebac
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/fpm/php-fpm.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/fpm/php-fpm.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/cli/conf.d/05-env.ini out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/cli/conf.d/05-env.ini has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/fpm/conf.d/05-env.ini out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/fpm/conf.d/05-env.ini has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO /etc/postfix/main.cf has md5sum 9292defe4b6014e06211f757095d986c should be 3ef849f32db5460ffefa05d0f1a25b44
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/postfix/main.cf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/postfix/main.cf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/postfix/sasl_passwd out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/postfix/sasl_passwd has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/cron.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/cron.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/php-fpm.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/php-fpm.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/nginx.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/nginx.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/postfix.conf out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/supervisor/conf.d/postfix.conf has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO /etc/php/7.1/mods-available/tideways.ini has md5sum c8fa775e7794665e7910a341e7f8a349 should be dc36b260ae6b18bccb15d297b1e78929
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/mods-available/tideways.ini out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/mods-available/tideways.ini has been updated
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO /etc/php/7.1/mods-available/xdebug.ini has md5sum c13c107d0ec91c574623ee2776b5b9d1 should be e67e39b9fb1c3150de1deb5c1124a100
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/mods-available/xdebug.ini out of sync
2017-07-27T10:17:25Z d4f93cc44884 confd[37]: INFO Target config /etc/php/7.1/mods-available/xdebug.ini has been updated
+ do_build_passona_inner
+ do_php_nginx_build_inner
+ do_build_user_ssh_keys
+ set +x
+ set +x
+ do_ubuntu_build_inner
+ :
+ do_build_permissions
+ '[' false '!=' true ']'
+ chown -R build:build /app/
+ do_assets_all
+ do_assets_download
+ '[' -n '' ']'
+ echo 'Skipping assets download due to ASSETS_PATH and/or ASSETS_S3_BUCKET_PATH not being set'
Skipping assets download due to ASSETS_PATH and/or ASSETS_S3_BUCKET_PATH not being set
Skipping assets apply due to ASSETS_PATH not being set
+ do_assets_apply
+ '[' -n '' ']'
+ echo 'Skipping assets apply due to ASSETS_PATH not being set'
+ do_composer
+ '[' -f /app/composer.json ']'
+ run_composer
+ '[' -n '' ']'
+ as_code_owner 'composer install --no-interaction --optimize-autoloader'
+ set +x
+ sudo -u build -E HOME=/home/build /bin/bash -c 'cd '\''/app'\''; composer install --no-interaction --optimize-autoloader'
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
    1/110:  https://codeload.github.com/sebastianbergmann/code-unit-reverse-lookup/legacy.zip/4419fcdb5eabb9caa61a27c7a1db532a6b55dd18
    2/110:  https://codeload.github.com/sebastianbergmann/php-file-iterator/legacy.zip/3cc8f69b3028d0f96a9078e6295d86e9bf019be5
...
sroze commented 7 years ago

Hey @richdynamix ! I think the bit to blame is the following:

        build:
            context: .
            args:
                - APP_ENV

Docker do not pick your environment variables for the build arguments unfortunately.

richdynamix commented 7 years ago

Ah this seems to fix my issue. Setting the values in docker-compose fixes it 👍