Environment variables in composer-override.yml doesn't get updated in Docker

[ronnyandre]

I have updated a container's environment variables, but it doesn't seem like those changes are being reflected in Docker (and Portainer as I use to control the variables).

I have tried to remove the container, rebuild it, removing the container's volume and recreating container. Nothing works. What am I doing wrong?

[Paraphraser]

Maybe a bit more detail would help - the contents of your docker-compose.yml and docker-compose.override.yml, which environment variables you are trying to change for which container.

Your subject line for this issue mis-spells docker-compose.override.yml. Is that just an error in the subject line or are you trying to use that file name too?

When you say that it isn't working, how do you know? For example, if I add the following to my NodeRed definition:

    environment:
      - RONNYANDRE=ISSUE241

then recreate the container (stop, rm, up), and then run

$ docker exec nodered bash -c 'echo $RONNYANDRE'
ISSUE241

That shows the variable is defined and is being transported inside the container but it means nothing to NodeRed so it does nothing. Maybe the variable you're defining is getting into the container but the process inside the container isn't reacting to it. I've mis-typed variable names a couple of times (particularly for InfluxDB where they are tricky to get right).

Anyway, a lot more information is likely to get a quicker answer!!

[ronnyandre]

Thanks for your feedback. However, I tried exactly what you said, to set evironment variable, ran docker-compose down CONTAINER, then docker-compose rm CONTAINER and then after changing environment variables, running docker-compose up -d CONTAINER. None of the environment variables were changed accordig to my compose-override.yml file.

So my actual issue is that the container named teslamate cannot connect to my Postgres database container, which is very strange, so I tried to tweak the environment settings.

I see that the changes in compose-override.yml isn't reflected in the docker-override.yml file, so that may be the issue.

Here is my compose-override.yml:

services:
  teslamate:
    container_name: teslamate
    image: teslamate/teslamate:latest
    restart: always
    environment:
      - DATABASE_USER=[redacted]
      - DATABASE_PASS=[redacted]
      - DATABASE_NAME=[redacted]
      - DATABASE_HOST=[redacted]
      - MQTT_HOST=[redacted]
      - MQTT_USERNAME=[redacted]
      - MQTT_PASSWORD=[redacted]
    ports:
      - 4000:4000
    volumes:
      - ./volumes/teslamate/import:/opt/app/import
    cap_drop:
      - all
  kanboard:
    container_name: kanboard
    image: kanboard/kanboard:latest
    restart: always
    ports:
      - 8088:80
      - 4433:443
    volumes:
      - ./volumes/kanboard/data:/var/www/app/data
      - ./volumes/kanboard/plugins:/var/www/app/plugins
      - ./volumes/kanboard/ssl:/etc/nginx/ssl
    environment:
      - PLUGIN_INSTALLER=true

And my complete docker-compose.yml:

version: '3.6'
services:
  teslamate:
    container_name: teslamate
    image: teslamate/teslamate:latest
    restart: always
    environment:
    - DATABASE_USER=[redacted]
    - DATABASE_PASS=[redacted]
    - DATABASE_NAME=[redacted]
    - DATABASE_HOST=[redacted]
    - MQTT_HOST=[redacted]
    - MQTT_USERNAME=[redacted]
    - MQTT_PASSWORD=[redacted]
    ports:
    - 4000:4000
    volumes:
    - ./volumes/teslamate/import:/opt/app/import
    cap_drop:
    - all
  kanboard:
    container_name: kanboard
    image: kanboard/kanboard:latest
    restart: always
    ports:
    - 8088:80
    - 4433:443
    volumes:
    - ./volumes/kanboard/data:/var/www/app/data
    - ./volumes/kanboard/plugins:/var/www/app/plugins
    - ./volumes/kanboard/ssl:/etc/nginx/ssl
    environment:
    - PLUGIN_INSTALLER=true
  influxdb:
    container_name: influxdb
    image: "influxdb:latest"
    restart: unless-stopped
    ports:
    - "8086:8086"
    - "8083:8083"
    - "2003:2003"
    environment:
    - INFLUXDB_HTTP_FLUX_ENABLED=false
    - INFLUXDB_REPORTING_DISABLED=false
    - INFLUXDB_HTTP_AUTH_ENABLED=false
    - INFLUX_USERNAME=[redacted]
    - INFLUX_PASSWORD=[redacted]
    - INFLUXDB_UDP_ENABLED=false
    - INFLUXDB_UDP_BIND_ADDRESS=0.0.0.0:8086
    - INFLUXDB_UDP_DATABASE=udp
    volumes:
    - ./volumes/influxdb/data:/var/lib/influxdb
    - ./backups/influxdb/db:/var/lib/influxdb/backup
    networks:
    - iotstack_nw

  nodered:
    container_name: nodered
    build: ./services/nodered/.
    restart: unless-stopped
    user: "0"
    privileged: true
    environment:
    - TZ=Etc/UTC
    ports:
    - "1880:1880"
    volumes:
    - ./volumes/nodered/data:/data
    - /var/run/docker.sock:/var/run/docker.sock
    - /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket
    devices:
    - "/dev/ttyAMA0:/dev/ttyAMA0"
    - "/dev/vcio:/dev/vcio"
    - "/dev/gpiomem:/dev/gpiomem"
    networks:
    - iotstack_nw
  portainer-ce:
    container_name: portainer-ce
    image: portainer/portainer-ce
    restart: unless-stopped
    ports:
    - "8000:8000"
    - "9000:9000"
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - ./volumes/portainer-ce/data:/data
  postgres:
    container_name: postgres
    image: postgres
    restart: unless-stopped
    environment:
    - POSTGRES_USER=[redacted]
    - POSTGRES_PASSWORD=[redacted]
    - POSTGRES_DB=[redacted]
    ports:
    - "5432:5432"
    volumes:
    - ./volumes/postgres/data:/var/lib/postgresql/data
    networks:
    - iotstack_nw
  grafana:
    container_name: grafana
    image: grafana/grafana
    restart: unless-stopped
    user: "0"
    ports:
    - "3000:3000"
    environment:
    - GF_PATHS_DATA=/var/lib/grafana
    - GF_PATHS_LOGS=/var/log/grafana
    volumes:
    - ./volumes/grafana/data:/var/lib/grafana
    - ./volumes/grafana/log:/var/log/grafana
    networks:
    - iotstack_nw

networks:
  iotstack_nw: # Exposed by your host.
    # external: true
    name: IOTstack_Net
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 10.77.60.0/24
        # - gateway: 10.77.60.1

  iotstack_nw_internal: # For interservice communication. No access to outside
    name: IOTstack_Net_Internal
    driver: bridge
    internal: true
    ipam:
      driver: default
      config:
      - subnet: 10.77.76.0/24
        # - gateway: 10.77.76.1
  vpn_nw: # Network specifically for VPN
    name: IOTstack_VPN
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 10.77.88.0/24
        # - gateway: 192.18.200.1

  nextcloud_internal: # Network for NextCloud service
    name: IOTstack_NextCloud
    driver: bridge
    internal: true

  # default:
  #   external: true
  #   name: iotstack_nw

  # hosts_nw:
  #   driver: hosts

[Schnuecks]

After you've create or modified compose-override.yml you need to run menu.sh again so the changes get merged, or you edit docker-compose.yml directly.

[Paraphraser]

@Schnuecks First off, I don't use the new menu (yet) and I've never tried using a compose-override so it's entirely possible that I have no idea what I am talking about BUT everything I have read says that docker-compose looks for both docker-compose.yml and compose-override.yml and it (docker-compose) does the merging.

See the last line of the documentation where it says:

The configuration in the docker-compose.override.yml file is applied over and in addition to the values in the docker-compose.yml file.

Is this wrong?

---

@ronnyandre There is no such thing as "down" followed by a container name. The sequence is:

$ docker-compose stop CONTAINER
$ docker-compose rm -f CONTAINER
$ docker-compose up -d CONTAINER

A "stop" followed by the "rm -f" does what a "down" does for each running container. The "down" also removes networks.

If you just stop then up (or stop then start), it resumes the stopped container. The "rm -f" removes the stopped container. Then, the next "up" causes the container to be re-created from the image.

I've defined a bunch of aliases/shell functions to speed up things like this. See my gist. In my case:

$ RECREATE teslamate

does the stop, rm and up. If I just wanted teslamate down, I'd:

$ TERMINATE teslamate

then, when I wanted to bring it up again:

$ UP teslamate

---

Teslamate can't communicate with PostgreSQL.

If you look at the definition of postgresql, you will see:

    networks:
    - iotstack_nw

which means postgresql is attached to that network. There is no "networks" definition for teslamate so it gets the default network which I think is still "iotstack_default" (old or new menu). If you just add that same "networks" definition to telslamate it will probably work.

Under old menu, "iotstack_default" is the only network so everything "just works".

I do not understand what problem Slyke is trying to solve with this explosion of networks under new menu but, in my view, it is a retrograde step. The average IOTstack user is not a comms guru and should not need to be. I spent years working in the data-communications field so I probably do count as something approximating a guru but this is something I would not have foisted on the IOTstack world. Not on a bet! In my view, any problem for which an explosion of networks and static IP ranges is the solution should be defined as completely out-of-scope for IOTstack. The cure is worse than the disease. The people who need this kind of advanced stuff should fork the repo and deal with it somewhere else. They should not expect all the poor suckers who simply want a quiet life to live with this extra baggage. It's bad and it's mean!

---

So, back to the original problem. If sticking teslamate into the same network as Postgres doesn't solve the problem then you should try the "docker exec" trick to display environment variables to make sure they're getting into the container.

Hope this helps (and that all my editorialising isn't too distracting - the new menu changes really bug me).

[Paraphraser]

@Schnuecks - an experiment.

Starting position:

$ pwd
/home/pi/IOTstack

Does Node-Red know about the environment variable called "NEW_EXAMPLE"?

$ docker exec nodered bash -c 'echo "NEW_EXAMPLE=$NEW_EXAMPLE"'
NEW_EXAMPLE=

Answer = no. Here's a prototype docker-compose.override.yml:

$ cat docker-compose.override.yml.off 
version: '3.6'

services:

  nodered:
    environment:
      - NEW_EXAMPLE="testing one two three"

Enable that file:

$ mv docker-compose.override.yml.off docker-compose.override.yml

Node-Red is already running but send the "up" command and see what happens:

$ docker-compose up -d nodered
Recreating nodered ... done

If docker-compose had decided nothing had changed, it would have reported:

nodered is up-to-date

so saying "Recreating nodered" is a sign that docker-compose knows that the container needs to be re-created.

Does the new container know about that "NEW_EXAMPLE" environment variable?

$ docker exec nodered bash -c 'echo "NEW_EXAMPLE=$NEW_EXAMPLE"'
NEW_EXAMPLE="testing one two three"

You betcha. So, no trip to the menu was needed to apply the override file.

Am I missing something?

[Schnuecks]

Hi, @Paraphraser the common docker-compose works the way if a docker-compose.override.yml exists it's loaded afterwards.

but here the compose-override.yml is not a standard file, because of that if the menu.sh is run, the compose-override.yml get merged into the docker-compose.yml.

https://sensorsiot.github.io/IOTstack/Custom/

[Paraphraser]

@Schnuecks Discovered something else. My "UP" alias is defined as:

alias UP='docker-compose -f ~/IOTstack/docker-compose.yml up -d'

The "advantage" of that is that I can run it from anywhere without having to start with a cd ~/IOTstack. Turns out that that breaks the override process. To make it work, it would need another -f argument with the path to the override file. According to the doco, you can use multiple -f in sequence and they are applied in order.

---

So, you're telling me that there is a "compose-override.yml" and a "docker-compose.override.yml".

Yikes! I'm so glad I stuck with old-menu.

This, of course, invites the question of why menu needs to replicate docker-compose functionality?

Mutter, mutter....

[Schnuecks]

@Paraphraser

Yes, I didn't know why here was tried to reinvent the wheel.

You can provide so many yml with -f as you want.

But currently you need to run menu.sh after modifying the compose-override.yml so it's get merged and the run your commands.

[ronnyandre]

Thanks for your help guys. It seems like you have to run the menu script and build the stack to merge composer-override.yml into docker-compose.yml. Seems a bit bloated, but this is how it has been since I started using IOTstack ages ago.

Thanks again!

[Paraphraser]

I really don't see any point to any of this override stuff. I view the menu (old or new) as an excellent way of getting started. Full stop. After that, I just change docker-compose.yml (and Dockerfiles and anything else I need to tinker with in ./services) as and when the need arises. I like to see what is going on. The menu (old or new) is far too opaque while override files add a level of indirection that is unnecessary. Maybe this is because I'm an old gut who grew up with CLIs, starting with punched cards.

[Schnuecks]

@Paraphraser Hi, i see it the way you do. Override is nice on docker for manually override things to fit in in your environment so you don't have to touch the original files. I think the override file is a nice way here to modifiy your installation without the need to use / modify service files, but i don't really know why here the compose override was reinvented instead of using docker defaults and work with that. Then theres is no need for run the menu.sh over and over again, just modify the docker-compose.override.yml and up the stack.

[Slyke]

Just as an FYI, the override file was added in so that any changes would come along if you backed up and restored IOTstack. If you use the -f flag you need to ensure that you use it each time you up your IOTstack (or do what @Paraphraser did and alias it). The override file was a way simplify this process, but it wasn't intended to replace it.