Environment Variables are not sent to dependency containers

[memotoro]

Hello

Just checking the behaviour of the plugin v4.14.0 and see if that is something that could be changed.

I have 2 docker compose files.

  config:
    - docker-compose-dependencies.yaml
    - docker-compose-test.yaml

The docker-compose-dependencies.yaml contains 2 containers that are required for the test to run. Let's call it container api and container db. These two containers will require specific environment variables. These containers are using healthchecks to make sure containers are correctly working before accessing them.

The docker-compose-test.yaml contains only the container with the test framework (let's call it tests) to test the container api. This tests container also requires specific environment variables.

It looks like the plugin kicks off the dependencies on the file docker-compose-dependencies.yaml first (containers api and db), if the flag run_dependencies is set to true, then those containers are started with a command similar to the following:

docker-compose -f docker-compose-dependencies.yaml -f docker-compose-test.yaml  -p buildkite0123... up -d --scale tests=0 tests

Later, the plugin runs the testing container in the file docker-compose-test.yaml specified in the run property. It looks like the environment variables are only passed to the container specified in the run command, similar to the following:

docker-compose -f docker-compose-dependencies.yaml -f docker-compose-test.yaml  -p buildkite0123... run --name buildkite123.... -e ENV1 -ENV2 ... --rm tests

I wonder if the environment variables cannot be propagated to the command that triggers the dependencies also. When using multiple configuration files, other containers need also configuration fed via environment variables. However, if those environment variables are not passed to the command shown above, then those container will not start correctly as some environment variables are missing, hence healthchecks fail and then the whole testing suite will fail. It would be nice if the dependencies are also fed with a command similar to the following:

docker-compose -f docker-compose-dependencies.yaml -f docker-compose-test.yaml  -p buildkite0123... -e ENV1 -e ENV2 ... up -d --scale tests=0 tests

I have tried to use the different flags to propagate environment variables, but it doesn't propagate the environment variables that I have set in the env attribute of the plugin or in the env attribute of the buildkite step command.

I have also set the property run_dependencies as false. In this case, the plugin starts all containers at the same time but only feed the environments to the container specified in the run property.

Is this the correct or expected behaviour ?

Thanks for any advice.

Tagging @toote @pzeballos as most recent contributors.

[toote]

@memotoro sorry for the delay in getting back to you!

I don't think that it would be easy to add to the plugin a way to specify what environment variables should be shared with each container. Luckily, docker compose has a way to do that. Have you tried adding environment stanzas to the services themselves? That way, if the variables are available in the context they will be used by docker compose and would not need any modification of the plugin. It would also serve as documentation on your services and dependencies as to what each need :)

[memotoro]

Hi @toote , thanks for the reply. I had the other docker-compose files with environment section per service. I also have the env section in buildkite with the definition of the ENV and values. However, that doesn't work either. It looks like with the plugin is kicking off the docker compose -f ..... up -d --scale tests=0 tests doesn't resolve ENVs defined in the env from buildkite, it only works if the envs in the environment section of the docker-compose file are hardcoded, but that defeat the purpose on some ENVs. I tried to propagate with buildkite flags but that doesn't work either. I'm using docker directly on bash for the time being, but wondering why the plugin is not passing the envs when doing the scale to 0.

[toote]

@memotoro sorry for the time it took me to get back to you but unfortunately I have not been able to duplicate your issue.

I used the following composefile:

services:
  db:
    image: bash:4.2
    command: bash -c 'env; sleep 60'
    environment:
      - TEST_VARIABLE2
      - TEST_VARIABLE3
  app:
    image: ruby
    depends_on:
      - db
    environment:
      - TEST_VARIABLE

And the following step in the pipeline:

steps:
  - label: test
    command: env
    plugins:
      - docker-compose#v4.14.0:
          run: app
          config: compose.yaml
          shell: false
          upload-container-logs: always

If the agent (or pipeline) I defined the variables TEST_VARIABLE and TEST_VARIABLE3 and the output of the step was:

$ docker-compose -f compose.yaml -p buildkite018d80c7a0374967a1aaf7f14effa393 up -d --scale app=0 app
Creating network "buildkite018d80c7a0374967a1aaf7f14effa393_default" with the default driver
Creating buildkite018d80c7a0374967a1aaf7f14effa393_db_1 ... done

:docker: Running 'env' in service app
$ docker-compose -f compose.yaml -p buildkite018d80c7a0374967a1aaf7f14effa393 run --name buildkite018d80c7a0374967a1aaf7f14effa393_app_build_31 [SNIP] --rm app env
Creating buildkite018d80c7a0374967a1aaf7f14effa393_app_run ... done
PATH=/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=b6ee35d65eb2
TERM=xterm
TEST_VARIABLE=testvalue
LANG=C.UTF-8
RUBY_MAJOR=3.2
RUBY_VERSION=3.2.2
RUBY_DOWNLOAD_SHA256=4b352d0f7ec384e332e3e44cdbfdcd5ff2d594af3c8296b5636c710975149e23
GEM_HOME=/usr/local/bundle
BUNDLE_SILENCE_ROOT_WARNING=1
BUNDLE_APP_CONFIG=/usr/local/bundle
HOME=/root

And the step had an artifact with the logs for the db dependency:

2024-02-06T23:36:31.561956162Z HOSTNAME=8d57b4ee6d15
2024-02-06T23:36:31.561987242Z _BASH_BASELINE=4.2.53
2024-02-06T23:36:31.561991873Z TEST_VARIABLE3=testvalue3
2024-02-06T23:36:31.561995257Z _BASH_BASELINE_PATCH=53
2024-02-06T23:36:31.561998408Z PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-02-06T23:36:31.562002152Z PWD=/
2024-02-06T23:36:31.562005214Z _BASH_VERSION=4.2.53
2024-02-06T23:36:31.562008198Z HOME=/root
2024-02-06T23:36:31.562011533Z SHLVL=1
2024-02-06T23:36:31.562014557Z _BASH_LATEST_PATCH=53
2024-02-06T23:36:31.562017612Z _=/usr/bin/env

As you can see TEST_VARIABLE is available and correctly defined in the main container being run and TEST_VARIABLE3 in the dependency container (but not TEST_VARIABLE2 because it was not defined) despite the plugin being run with no modifications. So I am not sure why a similar setup is not working for you :(